Agent routing benchmark

Multi-agent tasks: which coding agent

Four coding-agent CLIs ranked for five kinds of task, from public leaderboards and the outcomes of our own jobs. Refreshed weekly.

Generated 2026-10-14
Weekly refresh
5 task classes
6 kinds of coding and agentic work
4 agent CLIs

For agents: /bench.md (compact) · /bench.json (full) · /llms.txt

Which agent to use, by task class and kind of work · score 0–1 · Proven = at least 5 own jobs in the last 28 days
Task class	First choice	Score	Runner-up	Evidence
Codingrepo-level code changes	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.86	Codex CLIGPT-6.1 Sol xhigh (config) cost tier 3 of 4 0.65	Proven · 46 jobs
Coding: Bug fixingfinding and fixing defects in existing code	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.55	Antigravity CLIGPT-OSS 120B medium cost tier 1 of 4 0.57	Proven · 13 jobs
Coding: Features and refactorsnew features, tools and restructuring	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.79	Antigravity CLIClaude Opus 5.5 high cost tier 4 of 4 0.67	Proven · 18 jobs
Coding: Frontend and UIweb pages, UI components and styling	Claude CodeClaude Opus 5.5 high cost tier 4 of 4	0.68	Antigravity CLIClaude Opus 5.5 high cost tier 4 of 4 0.67	Unproven
Coding: Tests and reviewcode review, audits and tests	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.72	Codex CLIGPT-6.1 Sol xhigh (config) cost tier 3 of 4 0.52	Proven · 13 jobs
Agenticterminal and multi-step tool work	Claude CodeClaude Opus 5.5 high cost tier 4 of 4	0.71	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4 0.74	Unproven
Agentic: Terminal and sysadminshell, installs, configuration, server upkeep	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.73	Claude CodeClaude Opus 5.5 high cost tier 4 of 4 0.70	Unproven
Agentic: Tools and connectorsmulti-step work through APIs and tool connectors	Codex CLIGPT-6.1 Sol xhigh (config) cost tier 3 of 4	0.71	Claude CodeClaude Opus 5.5 high cost tier 4 of 4 0.57	Unproven
Researchweb research and fact-finding	Claude CodeClaude Fable 5.1 medium cost tier 4 of 4	0.54	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4 0.49	Proven · 30 jobs
Readinglong documents, transcripts, summaries, extraction	Claude CodeClaude Sonnet 5.5 high cost tier 3 of 4	0.65	Claude CodeClaude Fable 5.1 medium cost tier 4 of 4 0.65	Proven · 5 jobs
Writingdrafting and editing prose	Claude CodeClaude Opus 5.5 high cost tier 4 of 4	0.69	Antigravity CLIClaude Opus 5.5 high cost tier 4 of 4 0.67	Unproven · 2 jobs

Proven at least 5 own jobs of this kind in the last 28 days
Unproven fewer than 5 own jobs: ranked on public scores
cost tier 1 of 4 to cost tier 4 of 4 list-price cost tier, cheapest to dearest

Full ranking by task class

Top 8 per class. Score is the priority (0–1); Proven = at least 5 own jobs in 28 days, and an unproven entry leads only on a clear public margin; quality comes from a weekly random sample of jobs graded against their own instructions. Rows tagged tie or evidence were reordered by the rules in Method.

Coding
repo-level code changesswe_bench_verified_resolvedswe_bench_multilingual_resolvedterminal_bench_4_accuracyaa_coding
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Sonnet 5.5 high	Proven	0.86	0.94	0.91	46	4.0	1.00	cost tier 3 of 4 0.41
2	Codex CLI	GPT-6.1 Sol xhigh (config)	Proven tie	0.65	0.87	0.75	12	9.9	0.00	cost tier 3 of 4 0.45
3	Claude Code	Claude Opus 5.5 high	Unproven	0.67	1.00	0.50 default	2	9.0	0.33	cost tier 4 of 4 0.33
4	Claude Code	Claude Fable 5.1 medium	Unproven tie	0.62	0.87	0.50 default	4	4.8	0.67	cost tier 4 of 4 0.00
5	Antigravity CLI	Claude Sonnet 5.5 high	Unproven evidence	0.60	0.75 ×0.8	0.50 default	0	–	0.50	cost tier 3 of 4 0.39
6	Antigravity CLI	Claude Opus 5.5 high	Unproven evidence	0.59	0.80 ×0.8	0.50 default	0	–	0.50	cost tier 4 of 4 0.23
7	Antigravity CLI	GPT-OSS 120B medium	Unproven	0.57	0.50	0.50 default	0	–	0.50	cost tier 1 of 4 1.00
8	Antigravity CLI	Gemini 3.8 Flash high	Unproven tie	0.35	0.13	0.50 default	0	–	0.50	cost tier 2 of 4 0.61
	2 more candidates ranked in /bench.json

Kinds of work (ranked in the summary above): Bug fixing swe_bench_verified_resolved · Features and refactors arena_text_coding · Frontend and UI arena_webdev_overall · Tests and review own jobs only

Agentic
terminal and multi-step tool workterminal_bench_4_accuracyaa_agentic
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Opus 5.5 high	Unproven evidence	0.71	1.00	0.50 default	0	–	0.50	cost tier 4 of 4 0.42
2	Claude Code	Claude Sonnet 5.5 high	Unproven	0.74	0.94	0.50 default	0	–	0.50	cost tier 3 of 4 0.74
3	Codex CLI	GPT-6.1 Sol xhigh (config)	Unproven	0.70	0.87	0.50 default	0	–	0.50	cost tier 3 of 4 0.69
4	Claude Code	Claude Fable 5.1 medium	Unproven evidence	0.59	0.87	0.50 default	0	–	0.50	cost tier 4 of 4 0.00
5	Antigravity CLI	Claude Sonnet 5.5 high	Unproven tie	0.62	0.75 ×0.8	0.50 default	0	–	0.50	cost tier 3 of 4 0.55
6	Antigravity CLI	Claude Opus 5.5 high	Unproven	0.59	0.80 ×0.8	0.50 default	0	–	0.50	cost tier 4 of 4 0.23
7	Antigravity CLI	Gemini 3.8 Flash high	Unproven tie	0.41	0.13	0.50 default	0	–	0.50	cost tier 2 of 4 1.00
8	Grok CLI	Grok 4.6 medium (config)	Unproven	0.39	0.15	0.50 default	0	–	0.50	cost tier 3 of 4 0.84
	1 more candidate ranked in /bench.json

Kinds of work (ranked in the summary above): Terminal and sysadmin terminal_bench_4_accuracy arena_agent_bash_reliability_no_prereq_overall arena_agent_bash_recovery_steps_overall · Tools and connectors arena_agent_tool_hallucination_overall arena_agent_task_outcome_explicit_overall

Research
web research and fact-findingdrb_total_scorearena_search_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Fable 5.1 medium	Proven evidence	0.54	0.50 none	0.87	30	2.0	0.67	cost tier 4 of 4 0.00
2	Claude Code	Claude Sonnet 5.5 high	Proven evidence	0.49	0.50 none	0.86	22	5.0	0.33	cost tier 3 of 4 0.00
3	Codex CLI	GPT-6.1 Sol xhigh (config)	Proven evidence	0.38	0.50 none	0.59	32	12.6	0.00	cost tier 3 of 4 0.05
4	Grok CLI	Grok 4.5 medium (config)	Unproven evidence	0.47	0.50	0.50 default	0	–	0.50	cost tier 3 of 4 0.31
5	Claude Code	Claude Haiku 5.5 high	Unproven	0.65	0.50 none	0.50 default	2	0.4	1.00	cost tier 1 of 4 1.00

Reading
long documents, transcripts, summaries, extractionarena_document_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Sonnet 5.5 high	Proven tie	0.65	0.50 none	0.50 default	5	1.3	1.00	cost tier 3 of 4 1.00
2	Claude Code	Claude Fable 5.1 medium	Unproven	0.65	1.00	0.50 default	1	2.4	0.50	cost tier 4 of 4 0.00
3	Codex CLI	GPT-6.1 Sol xhigh (config)	Proven	0.51	0.50 none	0.72	18	9.5	0.00	cost tier 3 of 4 0.72
4	Grok CLI	Grok 4.6 medium (config)	Unproven tie	0.33	0.01	0.50 default	0	–	0.50	cost tier 3 of 4 0.82
5	Grok CLI	Grok 4.5 medium (config)	Unproven	0.32	0.00	0.50 default	0	–	0.50	cost tier 3 of 4 0.82

Writing
drafting and editing prosearena_text_creative_writingarena_text_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Opus 5.5 high	Unproven tie	0.69	1.00	0.50 default	2	2.4	0.60	cost tier 4 of 4 0.20
2	Antigravity CLI	Claude Opus 5.5 high	Unproven evidence	0.67	1.00	0.50 default	0	–	0.50	cost tier 4 of 4 0.10
3	Claude Code	Claude Fable 5.1 medium	Unproven evidence	0.66	0.93	0.50 default	1	1.6	0.80	cost tier 4 of 4 0.00
4	Claude Code	Claude Sonnet 5.5 high	Proven evidence	0.56	0.72	0.50 default	7	3.3	0.40	cost tier 3 of 4 0.32
5	Antigravity CLI	Gemini 3.8 Flash high	Unproven evidence	0.59	0.87	0.50 default	2	6.9	0.00	cost tier 2 of 4 0.47
6	Antigravity CLI	Claude Sonnet 5.5 high	Unproven evidence	0.56	0.72	0.50 default	0	–	0.50	cost tier 3 of 4 0.26
7	Grok CLI	Grok 4.5 medium (config)	Unproven evidence	0.53	0.60	0.50 default	0	–	0.50	cost tier 3 of 4 0.39
8	Codex CLI	GPT-6.1 Sol xhigh (config)	Proven evidence	0.47	0.58	0.50 default	6	3.8	0.20	cost tier 3 of 4 0.37
	4 more candidates ranked in /bench.json

Method

priority = 0.45*public_norm + 0.25*own_success + 0.15*speed_rank + 0.15*cost_rank: weights public 0.45, own success 0.25, speed 0.15, cost 0.15; every input is 0–1.
Public: each class metric is min-max scaled across the engine/model pairs ranked here, then averaged; 0.5 when a model has none.
Harness discount: a score measured in another agent's harness counts at 0.8× (marked ×0.8), for example a Terminal-Bench run in Claude Code credited to Antigravity running the same model. The engine's own harness, or a model-only benchmark, counts in full.
Own success: quality of own jobs from a weekly random sample of jobs graded against their own instructions (pass 1, partial 0.5, fail 0) once a pair has enough graded jobs; until then the share of own jobs in the last 28 days that finished cleanly, or 0.5 with fewer than 10 jobs. Speed: rank by own median minutes, else Artificial Analysis output speed, else 0.5.
Cost: weight 0.15; the cheaper choice scores higher when quality is close. Cost per successful job comes from own jobs (3 or more in the class), else the model's list price times the engine's typical job size, else 0.5; it is ranked on a log scale, cheapest 1. The $ to $$$$ tier is the model's list price alone (3 input : 1 output tokens, four log-spaced bins).
Evidence first: Proven means at least 5 own jobs in the last 28 days. Scores within 0.03 tie (marked tie) and go to more own jobs, then lower cost, then speed. An unproven entry ranks above a proven one only on a clear public margin: more than 0.03 ahead and a higher public score (else marked evidence), never on cost or speed alone. Quality before cost: an unproven entry leads only with a public score within 0.05 of the best.
Effort: the label next to a model is the reasoning effort to run it at: the standing default (Claude Code Fable medium, others high; Codex and Grok their config; Antigravity the -high id), moved only when a public leaderboard scores another level at least 0.03 higher for that class.
Kinds of work: Coding and agentic subclasses are ranked the same way, on the public metrics that measure that kind of work and own jobs of that kind; a subclass with neither is not listed.
Only aggregate job counts, success rates and median minutes are published; no prompts, job details or grades. Model lists for Grok CLI are unverified.

Full ranking by task class

Coding

Agentic

Research

Reading

Writing

Method