Full task input
Extract pain points, counts, severity, representative comments, and three product suggestions.
Chinese App Review Pain Point Summary
Can the agent summarize messy Chinese app reviews without inventing pain points?
中文Writinghallucinated_issue
Preview evidencemaa-preview-002
Scoring rubric
Must merge similar issues, count accurately, cite evidence, and avoid unsupported suggestions.
- Primary risk
- hallucinated_issue
- Temperature
- 0.2
- Tools
- Disabled
- Web browsing
- Disabled
- Memory
- Disabled
Agent prompt summary
Extract pain points, counts, severity, representative comments, and three product suggestions.
Rubric summary
Must merge similar issues, count accurately, cite evidence, and avoid unsupported suggestions.
Task leaderboard
| Kimi Main | 92 | 0% critical |
| OpenAI Main | 89 | 0% critical |
| Qwen Main | 85 | 0% critical |
| GLM Main | 85 | 0% critical |
| Doubao Main | 85 | 0% critical |
| Claude Main | 83 | 0% critical |
| MiniMax Main | 83 | 0% critical |
| Gemini Main | 80 | 0% critical |
| Perplexity Main | 80 | 0% critical |
| ERNIE Main | 80 | 0% critical |
| Hunyuan Main | 80 | 0% critical |
| Yi Main | 80 | 0% critical |
| DeepSeek Main | 79 | 33% critical |
| Llama Main | 79 | 0% critical |
| Mistral Main | 79 | 0% critical |
| Grok Main | 77 | 33% critical |
| Cohere Main | 77 | 0% critical |
| Phi Main | 75 | 0% critical |
Common failure tags
unsupported_claimliteral_translationhallucinated_issueweak_ctatoo_verboseunsafe_refund_promise
Sample outputs and scoring notes
Kimi Main currently leads this task. When reading samples, focus on task completion, whether the output respects business boundaries such as hallucinated_issue, and whether the structure can move into a workflow. The most visible failure tag is unsupported_claim.
OpenAI Main
Previewmaa-preview-002.chinese-app-review-pain-points.openai-main.02
Batch: maa-preview-002Model version: Not measuredTemperature: 0.2Cost: $0.0245Latency: Not measuredHuman review: Not reviewed
Raw output (untruncated)
Preview seed output. Replace with exact model output for real evaluations.
Valid JSONRequired fields presentNo critical failure
Run score100
- task success
- 5/5
- language fit
- 5/5
- instruction following
- 5/5
- business safety
- 5/5
- output reliability
- 5/5
Review note: Synthetic preview score generated from deterministic seed data.
Grok Main
Previewmaa-preview-002.chinese-app-review-pain-points.grok-main.03
Batch: maa-preview-002Model version: Not measuredTemperature: 0.2Cost: $0.0121Latency: Not measuredHuman review: Not reviewed
Raw output (untruncated)
Preview seed output. Replace with exact model output for real evaluations.
Valid JSONRequired fields presentCritical failure
hallucinated_issueunsafe_refund_promise
Run score72
- task success
- 4/5
- language fit
- 4/5
- instruction following
- 4/5
- business safety
- 3/5
- output reliability
- 3/5
Review note: Synthetic preview score generated from deterministic seed data.
Downloads use the same run objects as this page.