Agent routing benchmark

Multi-agent tasks: which coding agent

Four coding-agent CLIs ranked for five kinds of task, from public leaderboards and the outcomes of our own jobs. Refreshed weekly.

Generated 2026-10-13
Weekly refresh
5 task classes
6 kinds of coding and agentic work
4 agent CLIs

For agents: /bench.md (compact) · /bench.json (full) · /llms.txt

Which agent to use, by task class and kind of work · score 0–1 · Proven = at least 5 own jobs in the last 28 days
Task class	First choice	Score	Runner-up	Evidence
Codingrepo-level code changes	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.82	Codex CLIGPT-6.1 Sol cost tier 3 of 4 0.63	Proven · 44 jobs
Coding: Bug fixingfinding and fixing defects in existing code	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.55	Antigravity CLIGPT-OSS 120B cost tier 1 of 4 0.57	Proven · 13 jobs
Coding: Features and refactorsnew features, tools and restructuring	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.79	Antigravity CLIClaude Opus 5.5 cost tier 4 of 4 0.68	Proven · 17 jobs
Coding: Frontend and UIweb pages, UI components and styling	Claude CodeClaude Opus 5.5 cost tier 4 of 4	0.68	Antigravity CLIClaude Opus 5.5 cost tier 4 of 4 0.68	Unproven
Coding: Tests and reviewcode review, audits and tests	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.71	Codex CLIGPT-6.1 Sol cost tier 3 of 4 0.50	Proven · 12 jobs
Agenticterminal and multi-step tool work	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.68	Claude CodeClaude Opus 5.5 cost tier 4 of 4 0.68	Unproven
Agentic: Terminal and sysadminshell, installs, configuration, server upkeep	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4	0.68	Antigravity CLIClaude Sonnet 5.5 cost tier 3 of 4 0.65	Unproven
Agentic: Tools and connectorsmulti-step work through APIs and tool connectors	Codex CLIGPT-6.1 Sol cost tier 3 of 4	0.67	Claude CodeClaude Opus 5.5 cost tier 4 of 4 0.60	Unproven
Researchweb research and fact-finding	Codex CLIGPT-5.6 Sol cost tier 3 of 4	0.70	Claude CodeClaude Fable 5.1 cost tier 4 of 4 0.57	Unproven
Readinglong documents, transcripts, summaries, extraction	Claude CodeClaude Fable 5.1 cost tier 4 of 4	0.65	Claude CodeClaude Sonnet 5.5 cost tier 3 of 4 0.59	Unproven · 1 job
Writingdrafting and editing prose	Antigravity CLIGemini 3.8 Flash cost tier 2 of 4	0.67	Antigravity CLIClaude Opus 5.5 cost tier 4 of 4 0.68	Unproven

Proven at least 5 own jobs of this kind in the last 28 days
Unproven fewer than 5 own jobs: ranked on public scores
cost tier 1 of 4 to cost tier 4 of 4 list-price cost tier, cheapest to dearest

Full ranking by task class

Top 8 per class. Score is the priority (0–1); Proven = at least 5 own jobs in 28 days, and an unproven entry leads only on a clear public margin; quality comes from a weekly random sample of jobs graded against their own instructions. Rows tagged tie or evidence were reordered by the rules in Method.

Coding
repo-level code changesswe_bench_verified_resolvedswe_bench_multilingual_resolvedterminal_bench_4_accuracyaa_coding
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Sonnet 5.5	Proven	0.82	0.94	0.91	44	4.0	0.75	cost tier 3 of 4 0.36
2	Codex CLI	GPT-6.1 Sol	Proven tie	0.63	0.87	0.73	11	10.4	0.00	cost tier 3 of 4 0.38
3	Claude Code	Claude Opus 5.5	Unproven	0.66	1.00	0.50 default	2	9.0	0.25	cost tier 4 of 4 0.28
4	Claude Code	Claude Fable 5.1	Unproven tie	0.59	0.87	0.50 default	4	4.8	0.50	cost tier 4 of 4 0.00
5	Codex CLI	GPT-6 Sol	Unproven tie	0.58	0.70	0.50 default	0	–	0.50	cost tier 3 of 4 0.41
6	Antigravity CLI	Claude Sonnet 5.5	Unproven tie	0.60	0.75 ×0.8	0.50 default	0	–	0.50	cost tier 3 of 4 0.41
7	Antigravity CLI	Claude Opus 5.5	Unproven	0.60	0.80 ×0.8	0.50 default	0	–	0.50	cost tier 4 of 4 0.27
8	Antigravity CLI	GPT-OSS 120B	Unproven	0.57	0.50	0.50 default	0	–	0.50	cost tier 1 of 4 0.94
	8 more candidates ranked in /bench.json

Kinds of work (ranked in the summary above): Bug fixing swe_bench_verified_resolved · Features and refactors arena_text_coding · Frontend and UI arena_webdev_overall · Tests and review own jobs only

Agentic
terminal and multi-step tool workterminal_bench_4_accuracyaa_agentic
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Sonnet 5.5	Unproven tie	0.68	0.94	0.50 default	0	–	0.50	cost tier 3 of 4 0.35
2	Claude Code	Claude Opus 5.5	Unproven	0.68	1.00	0.50 default	0	–	0.50	cost tier 4 of 4 0.20
3	Codex CLI	GPT-6.1 Sol	Unproven	0.64	0.87	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
4	Codex CLI	GPT-6 Sol	Unproven tie	0.57	0.70	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
5	Antigravity CLI	Claude Sonnet 5.5	Unproven tie	0.59	0.75 ×0.8	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
6	Antigravity CLI	Claude Opus 5.5	Unproven tie	0.59	0.80 ×0.8	0.50 default	0	–	0.50	cost tier 4 of 4 0.18
7	Claude Code	Claude Fable 5.1	Unproven	0.59	0.87	0.50 default	0	–	0.50	cost tier 4 of 4 0.00
8	Codex CLI	GPT-5.6 Sol	Unproven	0.46	0.47	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
	6 more candidates ranked in /bench.json

Kinds of work (ranked in the summary above): Terminal and sysadmin terminal_bench_4_accuracy arena_agent_bash_reliability_no_prereq_overall arena_agent_bash_recovery_steps_overall · Tools and connectors arena_agent_tool_hallucination_overall arena_agent_task_outcome_explicit_overall

Research
web research and fact-findingdrb_total_scorearena_search_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Codex CLI	GPT-5.6 Sol	Unproven	0.70	1.00	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
2	Claude Code	Claude Fable 5.1	Proven evidence	0.57	0.50 none	0.83	24	2.0	0.80	cost tier 4 of 4 0.14
3	Claude Code	Claude Sonnet 5.5	Proven evidence	0.53	0.50 none	0.91	21	5.0	0.40	cost tier 3 of 4 0.14
4	Codex CLI	GPT-6.1 Sol	Proven evidence	0.41	0.50 none	0.61	31	12.6	0.00	cost tier 3 of 4 0.18
5	Claude Code	Claude Haiku 5.5	Unproven	0.65	0.50 none	0.50 default	2	0.4	1.00	cost tier 1 of 4 1.00
6	Codex CLI	GPT-6 Astra	Unproven tie	0.44	0.50 none	0.50 default	2	2.8	0.60	cost tier 4 of 4 0.00
7	Codex CLI	GPT-6 Sol	Unproven	0.43	0.50 none	0.50 default	1	6.9	0.20	cost tier 3 of 4 0.34
8	Grok CLI	Grok 4.5	Unproven	0.26	0.00	0.50 default	0	–	0.50	cost tier 3 of 4 0.40

Reading
long documents, transcripts, summaries, extractionarena_document_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Claude Code	Claude Fable 5.1	Unproven	0.65	1.00	0.50 default	1	2.4	0.50	cost tier 4 of 4 0.02
2	Claude Code	Claude Sonnet 5.5	Proven	0.59	0.50 none	0.50 default	5	1.3	1.00	cost tier 3 of 4 0.60
3	Codex CLI	GPT-6.1 Sol	Proven evidence	0.47	0.50 none	0.71	17	9.1	0.00	cost tier 3 of 4 0.45
4	Codex CLI	GPT-6 Sol	Unproven	0.53	0.50 none	0.50 default	1	1.8	0.75	cost tier 3 of 4 0.42
5	Codex CLI	GPT-5.6 Sol	Unproven	0.45	0.51	0.50 default	2	5.8	0.25	cost tier 3 of 4 0.42
6	Codex CLI	GPT-5.6 Luna	Unproven tie	0.39	0.08	0.50 default	0	–	0.50	cost tier 1 of 4 1.00
7	Codex CLI	GPT-5.6 Terra	Unproven	0.40	0.32	0.50 default	0	–	0.50	cost tier 3 of 4 0.39
8	Codex CLI	GPT-6 Astra	Unproven	0.32	0.26	0.50 default	0	–	0.50	cost tier 4 of 4 0.00
	2 more candidates ranked in /bench.json

Writing
drafting and editing prosearena_text_creative_writingarena_text_overall
#	Engine	Model	Evidence	Score	Public	Own success	Jobs	Median min	Speed	Cost
1	Antigravity CLI	Gemini 3.8 Flash	Unproven evidence	0.67	0.87	0.50 default	0	–	0.50	cost tier 2 of 4 0.55
2	Antigravity CLI	Claude Opus 5.5	Unproven evidence	0.68	1.00	0.50 default	0	–	0.50	cost tier 4 of 4 0.19
3	Claude Code	Claude Opus 5.5	Unproven tie	0.64	1.00	0.50 default	2	2.4	0.20	cost tier 4 of 4 0.21
4	Claude Code	Claude Fable 5.1	Unproven evidence	0.64	0.93	0.50 default	1	1.6	0.60	cost tier 4 of 4 0.01
5	Antigravity CLI	Claude Sonnet 5.5	Unproven evidence	0.58	0.72	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
6	Grok CLI	Grok 4.5	Unproven evidence	0.53	0.60	0.50 default	0	–	0.50	cost tier 3 of 4 0.40
7	Codex CLI	GPT-5.6 Sol	Unproven evidence	0.54	0.64	0.50 default	0	–	0.50	cost tier 3 of 4 0.34
8	Codex CLI	GPT-6.1 Sol	Proven evidence	0.50	0.58	0.50 default	5	2.0	0.40	cost tier 3 of 4 0.36
	10 more candidates ranked in /bench.json

Method

priority = 0.45*public_norm + 0.25*own_success + 0.15*speed_rank + 0.15*cost_rank: weights public 0.45, own success 0.25, speed 0.15, cost 0.15; every input is 0–1.
Public: each class metric is min-max scaled across the engine/model pairs ranked here, then averaged; 0.5 when a model has none.
Harness discount: a score measured in another agent's harness counts at 0.8× (marked ×0.8), for example a Terminal-Bench run in Claude Code credited to Antigravity running the same model. The engine's own harness, or a model-only benchmark, counts in full.
Own success: quality of own jobs from a weekly random sample of jobs graded against their own instructions (pass 1, partial 0.5, fail 0) once a pair has enough graded jobs; until then the share of own jobs in the last 28 days that finished cleanly, or 0.5 with fewer than 10 jobs. Speed: rank by own median minutes, else Artificial Analysis output speed, else 0.5.
Cost: weight 0.15; the cheaper choice scores higher when quality is close. Cost per successful job comes from own jobs (3 or more in the class), else the model's list price times the engine's typical job size, else 0.5; it is ranked on a log scale, cheapest 1. The $ to $$$$ tier is the model's list price alone (3 input : 1 output tokens, four log-spaced bins).
Evidence first: Proven means at least 5 own jobs in the last 28 days. Scores within 0.03 tie (marked tie) and go to more own jobs, then lower cost, then speed. An unproven entry ranks above a proven one only on a clear public margin: more than 0.03 ahead and a higher public score (else marked evidence), never on cost or speed alone.
Kinds of work: Coding and agentic subclasses are ranked the same way, on the public metrics that measure that kind of work and own jobs of that kind; a subclass with neither is not listed.
Only aggregate job counts, success rates and median minutes are published; no prompts, job details or grades. Model lists for Grok CLI are unverified.

Full ranking by task class

Coding

Agentic

Research

Reading

Writing

Method