Protocol only — no published results
Multilingual support triage benchmark protocol
A proposed protocol for measuring classification, escalation, format stability, and language-specific failure modes.
Benchmark protocols
Transparent evaluation designs. Preview protocols show no winner until reviewed results exist.
6source-linked records
Protocol only — no published results
Structured document extraction benchmark protocol
A proposed protocol for testing field accuracy, evidence spans, abstention, and schema compliance on business documents.
Protocol only — no published results
Repository change benchmark protocol
A proposed protocol for measuring accepted software changes, regressions, scope discipline, and review effort.
Protocol only — no published results
Creative image brief benchmark protocol
A proposed protocol for measuring mandatory brief adherence, visual defects, edit burden, provenance completeness, and reviewer disagreement.
Protocol only — no published results
Explainer video production benchmark protocol
A proposed protocol for testing script fidelity, scene continuity, audio and caption quality, edit burden, consent evidence, and final-review outcomes.
Protocol only — no published results
Agentic workflow execution benchmark protocol
A proposed protocol for measuring task completion, unsafe action prevention, duplicate effects, recovery, and human operating burden in sandboxed workflows.
How to use this index
Start with a task, open the cited sources, then run a dated pilot under your real constraints.
- ShortlistUse stable product fit—not popularity—as the first filter.
- VerifyCheck current provider documentation, terms, and regional availability.
- PilotRecord task inputs, model conditions, failures, and human review effort.