Where it should be considered first
- 中文 workflows, especially where this agent shows its strongest language score.
- Extraction tasks where it is a reasonable first shortlist candidate.
- Budget-sensitive automation pilots.
DeepSeek Main
Best value profile for structured extraction and classification.
DeepSeeklowArena #2
Preview evidence60 runs
- Provider
- DeepSeek
- Exact model version
- Not measured
- Evaluation batch
- maa-preview-002
- Test date
- 2026-06-27
- Context window
- Not measured
- Tool mode
- disabled
- Pricing source
- Not measured
- Latency
- Not measured
Best fit
- 中文 workflows where the agent has its strongest language evidence.
- Extraction tasks as a first shortlist candidate against at least two alternatives.
- Budget-sensitive, lower-risk internal pilots.
Not a fit for
- Critical-failure rate is 7%; do not fully automate refund, compliance, safety, or legal work first.
- Format pass is 100%; retest JSON, dates, and missing fields before extraction launch.
- Do not treat the current score as permanent after model, pricing, or tool changes.
Alternatives
Overall80
Pass rate70%
Critical7%
Format pass100%
Measured avg. cost$0.0050
LatencyNot measured
Review before production
- Review failure tags such as weak_cta, missing_field, hallucinated_issue before production use.
- Critical-failure rate is 7%; do not rely on the overall score alone for support, refund, or compliance workflows.
- Format pass rate is 100%; retest JSON and missing-field behavior before extraction use cases go live.
Compare next
Profile metrics
Overall score: 80 Win rate: 5% Pass rate: 70% Critical: 7% Format pass rate: 100% Average run cost: $0.0050
Common failure tags
weak_ctamissing_fieldhallucinated_issue
Language performance
| 中文 | 82 |
| English | 78 |
| 日本語 | 79 |
| Español | 80 |
Task type performance
| Support | 79 |
| Writing | 78 |
| Extraction | 83 |
Decision profile
How should this agent enter a real pilot?
Translate the score into practical selection guidance: fit, no-fit, pilot path, and task evidence.
- 中文 workflows where the agent has its strongest language evidence.
- Extraction tasks as a first shortlist candidate against at least two alternatives.
- Budget-sensitive, lower-risk internal pilots.
- Critical-failure rate is 7%; do not fully automate refund, compliance, safety, or legal work first.
- Format pass is 100%; retest JSON, dates, and missing fields before extraction launch.
- Do not treat the current score as permanent after model, pricing, or tool changes.
- Prepare 20-50 real samples across normal, edge, and high-risk cases.
- Compare against one high-score candidate and one lower-cost candidate.
- Record repair time, failure tags, and unacceptable failures before expanding automation.
Recommended pilot path
- Use 20–50 real samples across normal, edge, and high-risk inputs.
- Keep at least one lower-cost comparator.
- Human-review every critical failure and high-risk output.
- Retest after model, prompt, tool, or business-policy changes.
Task evidence
Representative task evidence for this agent
中文 / ExtractionChinese Contract Field Extraction85 · hallucinated_signing_date中文 / ExtractionChinese Sales Call Summary85 · missed_buying_signal日本語 / ExtractionJapanese Product Specification Extraction83 · hallucinated_materialEspañol / ExtractionSpanish Order Confirmation Extraction83 · wrong_date_formatEspañol / ExtractionSpanish Survey Insight Clustering83 · overmerged_feedback