Generated from batch: maa-preview-002
AAA.win tested 18 AI agents on 20 multilingual business tasks across 4 languages. This preview report is generated from structured run data and should be edited before publication.
AAA.win tested 18 AI agents on 20 multilingual business tasks across 4 languages. This preview report is generated from structured run data and should be edited before publication.
A fast read on winners, caveats, and signals that need human review before public claims.
Compares agents across Chinese, English, Japanese, and Spanish markets so teams do not choose from English-only evidence.
Focuses on critical failures, unsafe promises, invented fields, and unusable outputs.
Helps teams choose by cost, working language, and risk tolerance instead of a single average.
Explains which task families separate agents most: support, writing, and structured extraction.
Lists the conditions required before using results in a launch, article, or commercial page.
Use the global ranking to shortlist, then inspect language, task type, failure tags, and cost before piloting. High-risk workflows must return to task samples and human-review rules.
Support, writing, extraction, internal automation, or cross-border work.
Read task pages, failure tags, format pass, and critical-failure rate.
Retest with your own inputs and record human repair time.
Today's realistic scope is to improve evidence quality and localization depth across the existing 20-task arena, while keeping preview results clearly labeled.
Expected capacity today: 20 task pages can be reviewed for text quality; 8-10 can receive deeper evidence edits if each task gets careful prompt/rubric treatment.
| Rank | Agent | Score | Pass Rate | Critical Failure Rate | Cost Tier |
|---|---|---|---|---|---|
| 1 | Claude Main | 87 | 97% | 12% | premium |
| 2 | OpenAI Main | 86 | 92% | 12% | premium |
| 3 | Qwen Main | 84 | 93% | 10% | standard |
| 4 | Kimi Main | 82 | 87% | 12% | standard |
| 5 | GLM Main | 81 | 88% | 7% | standard |
| 6 | Mistral Main | 81 | 85% | 2% | standard |
| 7 | MiniMax Main | 80 | 90% | 2% | standard |
| 8 | Gemini Main | 80 | 82% | 12% | standard |
| 9 | Cohere Main | 80 | 77% | 12% | standard |
| 10 | DeepSeek Main | 80 | 70% | 7% | low |
| 11 | Hunyuan Main | 80 | 85% | 2% | standard |
| 12 | ERNIE Main | 79 | 78% | 3% | standard |
| 13 | Llama Main | 79 | 75% | 7% | low |
| 14 | Perplexity Main | 79 | 73% | 17% | standard |
| 15 | Doubao Main | 79 | 67% | 12% | low |
| 16 | Yi Main | 78 | 62% | 8% | low |
| 17 | Grok Main | 75 | 37% | 27% | standard |
| 18 | Phi Main | 75 | 37% | 20% | low |
| Language | Winner | Score | Critical Failure Rate |
|---|---|---|---|
| 中文 | Qwen Main | 89 | 7% |
| English | OpenAI Main | 93 | 7% |
| 日本語 | Claude Main | 89 | 13% |
| Español | Claude Main | 88 | 13% |
| Task Type | Winner | Score | Critical Failure Rate |
|---|---|---|---|
| Support | Claude Main | 90 | 13% |
| Writing | Claude Main | 90 | 11% |
| Extraction | Qwen Main | 88 | 6% |
| Failure Tag | Count |
|---|---|
| literal_translation | 84 |
| unsupported_claim | 84 |
| weak_cta | 46 |
| unsafe_refund_promise | 41 |
| missing_field | 38 |
| too_verbose | 36 |
| invalid_json | 31 |
| missed_dependency | 30 |
| generic_ai_copy | 24 |
| wrong_date_format | 19 |
| Task | Language | Type | Winner | Score | Primary Risk |
|---|---|---|---|---|---|
| Chinese Customer Complaint Triage | 中文 | Support | Qwen Main | 85 | unsafe_refund_promise |
| Chinese App Review Pain Point Summary | 中文 | Writing | Kimi Main | 92 | hallucinated_issue |
| Chinese Contract Field Extraction | 中文 | Extraction | Qwen Main | 96 | hallucinated_signing_date |
| Chinese Sales Call Summary | 中文 | Extraction | Qwen Main | 96 | missed_buying_signal |
| Chinese Invoice Dispute Reply | 中文 | Support | OpenAI Main | 85 | unauthorized_credit |
| SaaS Landing Page Hero Rewrite | English | Writing | OpenAI Main | 93 | generic_ai_copy |
| Meeting Notes Action Item Extraction | English | Extraction | OpenAI Main | 89 | discussion_as_action |
| Refund Policy Boundary Reply | English | Support | OpenAI Main | 96 | unsafe_refund_promise |
| English Security Questionnaire Answer | English | Support | OpenAI Main | 96 | unsupported_security_claim |
| English Churn Risk Email | English | Writing | Claude Main | 95 | tone_deaf_retention |
| Japanese Business Email Politeness Rewrite | 日本語 | Writing | OpenAI Main | 85 | unnatural_japanese |
| Japanese Appointment Intent Classification | 日本語 | Support | Claude Main | 92 | wrong_intent |
| Japanese Product Specification Extraction | 日本語 | Extraction | Qwen Main | 91 | hallucinated_material |
| Japanese Support Escalation Note | 日本語 | Support | Claude Main | 92 | lost_escalation_context |
| Japanese Pricing Page Localization | 日本語 | Writing | Claude Main | 92 | literal_pricing_copy |
| Spanish Support Reply for Wrong Item | Español | Support | Claude Main | 89 | unsafe_refund_promise |
| Spanish Ad Headline Localization | Español | Writing | Claude Main | 92 | literal_translation |
| Spanish Order Confirmation Extraction | Español | Extraction | Claude Main | 85 | wrong_date_format |
| Spanish Billing Cancellation Reply | Español | Support | Claude Main | 91 | wrong_cancellation_policy |
| Spanish Survey Insight Clustering | Español | Extraction | Qwen Main | 83 | overmerged_feedback |