Full task input
Extract owner, deadline, task, and risk from English beta launch meeting notes.
Meeting Notes Action Item Extraction
Can the agent distinguish real action items from general meeting discussion?
EnglishExtractiondiscussion_as_action
Preview evidencemaa-preview-002
Scoring rubric
Must use unclear when owner/deadline is absent and avoid turning discussions into tasks.
- Primary risk
- discussion_as_action
- Temperature
- 0.2
- Tools
- Disabled
- Web browsing
- Disabled
- Memory
- Disabled
Agent prompt summary
Extract owner, deadline, task, and risk from English beta launch meeting notes.
Rubric summary
Must use unclear when owner/deadline is absent and avoid turning discussions into tasks.
Task leaderboard
| OpenAI Main | 89 | 0% critical |
| Gemini Main | 85 | 0% critical |
| Claude Main | 83 | 0% critical |
| Qwen Main | 83 | 0% critical |
| Mistral Main | 83 | 0% critical |
| Cohere Main | 83 | 0% critical |
| GLM Main | 83 | 0% critical |
| Grok Main | 80 | 0% critical |
| DeepSeek Main | 80 | 0% critical |
| Llama Main | 80 | 0% critical |
| Kimi Main | 80 | 0% critical |
| Hunyuan Main | 80 | 0% critical |
| Perplexity Main | 79 | 33% critical |
| ERNIE Main | 79 | 0% critical |
| MiniMax Main | 79 | 0% critical |
| Doubao Main | 79 | 0% critical |
| Yi Main | 77 | 33% critical |
| Phi Main | 76 | 0% critical |
Common failure tags
unsupported_claimdiscussion_as_actionliteral_translationgeneric_ai_copyunsafe_refund_promisecitation_overreach
Sample outputs and scoring notes
OpenAI Main currently leads this task. When reading samples, focus on task completion, whether the output respects business boundaries such as discussion_as_action, and whether the structure can move into a workflow. The most visible failure tag is unsupported_claim.
OpenAI Main
Previewmaa-preview-002.meeting-notes-action-items.openai-main.02
Batch: maa-preview-002Model version: Not measuredTemperature: 0.2Cost: $0.0245Latency: Not measuredHuman review: Not reviewed
Raw output (untruncated)
Preview seed output. Replace with exact model output for real evaluations.
Valid JSONRequired fields presentNo critical failure
Run score100
- task success
- 5/5
- language fit
- 5/5
- instruction following
- 5/5
- business safety
- 5/5
- output reliability
- 5/5
Review note: Synthetic preview score generated from deterministic seed data.
Perplexity Main
Previewmaa-preview-002.meeting-notes-action-items.perplexity-main.01
Batch: maa-preview-002Model version: Not measuredTemperature: 0.2Cost: $0.0118Latency: Not measuredHuman review: Not reviewed
Raw output (untruncated)
Preview seed output. Replace with exact model output for real evaluations.
Valid JSONRequired fields presentCritical failure
discussion_as_actioncitation_overreach
Run score76
- task success
- 4/5
- language fit
- 4/5
- instruction following
- 4/5
- business safety
- 3/5
- output reliability
- 4/5
Review note: Synthetic preview score generated from deterministic seed data.
Downloads use the same run objects as this page.