English scores did not predict multilingual rank.
Several agents that looked strongest in English were weaker in Chinese support or Japanese business tone.
종합 순위
마케팅 문구가 아니라 다국어 비즈니스 성능으로 정렬합니다.
| Rank | Agent | Overall | Win rate | Pass rate | Critical | Best language | Best for | Cost |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Main Anthropic | 87 | 55% | 97% | 12% | English | 지원 | premium |
| 2 | OpenAI Main OpenAI | 86 | 35% | 92% | 12% | English | 작성 | premium |
| 3 | Qwen Main Alibaba | 84 | 25% | 93% | 10% | 中文 | 추출 | standard |
| 4 | Gemini Main | 80 | 0% | 82% | 12% | English | 추출 | standard |
| 5 | DeepSeek Main DeepSeek | 80 | 5% | 70% | 7% | 中文 | 추출 | low |
| 6 | Grok Main xAI | 75 | 0% | 37% | 27% | English | 작성 | standard |
핵심 발견
실무에 유용한 이야기는 항상 종합 1위와 같지 않습니다.
Support tasks exposed unsafe promises.
The biggest failures were often business-boundary failures, not grammar mistakes.
Japanese writing separated grammar from natural tone.
Correct Japanese was not enough. Natural, concise business phrasing mattered.
Extraction revealed the widest reliability gap.
Valid JSON, null handling, date formats, and missing-field discipline changed rankings.
Language Winners
Find the agent that wins the language you actually work in.
최고 中文
89
Qwen Main
최고 English
93
OpenAI Main
최고 日本語
89
Claude Main
최고 Español
88
Claude Main
Failure Modes
The most common failures were not always language errors. They were business risks.
literal_translation
26
프리뷰 실행
unsafe_refund_promise
23
프리뷰 실행
weak_cta
21
프리뷰 실행
unsupported_claim
17
프리뷰 실행
invalid_json
13
프리뷰 실행
Task Evidence
Every score should lead back to prompts, rubrics, outputs, and failure tags.
Chinese Customer Complaint Triage
주요 위험: unsafe_refund_promise
80
승자: Qwen Main
unsafe_refund_promise
Chinese App Review Pain Point Summary
주요 위험: hallucinated_issue
82
승자: OpenAI Main
hallucinated_issue
Chinese Contract Field Extraction
주요 위험: hallucinated_signing_date
82
승자: Qwen Main
hallucinated_signing_date
Chinese Sales Call Summary
주요 위험: missed_buying_signal
84
승자: Qwen Main
missed_buying_signal
Chinese Invoice Dispute Reply
주요 위험: unauthorized_credit
80
승자: OpenAI Main
unauthorized_credit
SaaS Landing Page Hero Rewrite
주요 위험: generic_ai_copy
83
승자: OpenAI Main
generic_ai_copy
Agent Profiles
Each profile reflects Multilingual Agent Arena #2, not a universal model ranking.
Claude Main
Strong writing and safety boundaries, especially in support tasks.
87
too_verboseoverly_humbleunsafe_refund_promise
OpenAI Main
Strong generalist with balanced writing and support safety.
86
missed_dependencygeneric_ai_copyunsafe_refund_promise
Qwen Main
Strong Chinese business language and structured extraction.
84
literal_translationunnatural_japaneseunauthorized_credit
Gemini Main
Reliable extraction profile with mixed localization performance.
80
literal_translationwrong_date_formatunsafe_refund_promise
DeepSeek Main
Best value profile for structured extraction and classification.
80
weak_ctamissing_fieldhallucinated_issue
Grok Main
Fast outputs with higher variance on business constraints.
75
unsafe_refund_promiseunsupported_claiminvalid_json