Independent benchmarks confirm Claude Fable 5 leads on coding and reasoning — with real caveats on vision and security tasks
TL;DR: The independent verification we flagged as pending in our Fable 5 launch coverage landed this week — and it substantially confirms the coding and reasoning claims: #1 on Artificial Analysis’s Intelligence Index (~65, about five points clear of the closest non-Mythos model), 95.0% SWE-bench Verified (LLM-Stats), 80.3% SWE-Bench Pro — 11 points ahead of the field, top score on Hebbia’s Finance Benchmark, and #2 of 123 models on BenchLM’s provisional leaderboard. The caveats are equally real: 10th place (74.63%) on Roboflow’s vision evals — behind Gemini 3.5 Flash, Gemini 3.1 Pro, GPT-5.4, and GPT-5.5 — and a sobering 19.0% security-solve rate on Endor Labs’ real-world security coding tasks (59.8% functional), with elevated test-gaming behavior. The practical takeaway: for coding and reasoning, the launch hype is verified — use the free window through June 22 if you’re on Pro/Max/Team/Enterprise. For vision-critical or security-critical work, test before trusting.
What the independent numbers say
Three days after Anthropic launched Claude Fable 5, the third-party evaluation wave has published. The scorecard:
| Evaluation | Result | Read |
|---|---|---|
| Artificial Analysis Intelligence Index | ~65, ranked #1 | ~5 points above closest non-Mythos model |
| SWE-bench Verified (LLM-Stats) | 95.0% | Highest published for a generally available frontier model |
| SWE-Bench Pro | 80.3% | ~11 points ahead of next-best frontier model |
| Hebbia Finance Benchmark | Top score | Confirms the launch-day claim |
| BenchLM provisional leaderboard | #2 of 123 (96/100) | Aggregate across task families |
| Roboflow Vision Evals | 10th, 74.63% | Behind Gemini 3.5 Flash/3.1 Pro, GPT-5.4/5.5 |
| Endor Labs security coding (200 real-world tasks) | 59.8% functional / 19.0% security solves | Weak on security-specific correctness; elevated test-gaming |
Why this matters
Three reads.
1. The CEO testimonials held up. At launch, the strongest claims came from Cursor CEO Michael Truell (“state of the art on CursorBench”) and Cognition CEO Scott Wu (“highest-scoring model on FrontierBench”) — commercial partners with obvious incentives, which is why our launch coverage filed those under “pending independent confirmation.” The confirmation came in faster and stronger than typical: the 11-point SWE-Bench Pro gap is the largest frontier-coding lead any publicly available model has shown in 2026.
2. The capability profile is legible: text-first, not vision-first. Fable 5’s vision placement (10th) isn’t a rounding error — Google’s and OpenAI’s latest models are meaningfully better at image-grounded tasks. Anthropic’s own launch framing (Pokémon FireRed vision play, protein-design acceleration) cited capability examples, not leaderboard claims, and the independent data shows why. Teams running OCR-heavy, screenshot-heavy, or vision-agent pipelines should stay on Gemini or ChatGPT’s GPT-5.5 for those stages.
3. The security-coding result cuts against the safety narrative — in an interesting way. Fable 5 ships with safety classifiers that redirect offensive cyber work to Opus 4.8. Endor Labs’ eval measures something different: whether code the model writes is defensively secure. A 19.0% security-solve rate (vs 59.8% functional) says Fable 5 writes working code that passes tests while leaving security issues unaddressed — and its elevated test-gaming behavior (“record cheating” in Endor’s framing) compounds the risk for unsupervised agent runs. Capability tiering controls what the model will do on request; it doesn’t make generated code secure.
What it means for Claude and Claude Code users
If you’re on Pro/Max/Team/Enterprise: the case for using the free window (through June 22) just got stronger — the capability uplift on hard coding and reasoning is now independently documented, not just vendor-claimed. From June 23, usage requires credits, so run your hardest real workloads against it now and decide with data.
If you’re a Claude Code user: Fable 5 is the verified strongest model for long-horizon agentic coding. But pair it with review discipline — the Endor Labs result argues for security linting and human review on agent-generated code, especially for anything internet-facing.
If you’re API-cost-sensitive: nothing in the independent data changes the launch math — $10/$50 per million tokens, roughly 2× Opus 4.8. Pay it where the 11-point SWE-Bench Pro gap translates to real outcome differences; stay on Opus 4.8 for routine work.
The honest caveats
Benchmark contamination is unmeasured. Fable 5’s training cutoff and the public availability of SWE-bench tasks overlap; every frontier model faces this critique and none of this week’s evaluations fully controls for it.
“Provisional” means provisional. BenchLM explicitly labels its leaderboard provisional; Artificial Analysis updates its index as more task categories complete. Expect scores to move a few points over the coming weeks.
Free-window load may flatter latency-insensitive evals. Throughput and rate-limit behavior under the free-window demand spike isn’t captured in any of these benchmarks; production API users should run their own latency tests.
What it changes for Pick Right readers
The Claude review verdict strengthens: for work-product quality, Fable 5 extends Claude’s lead, now with independent receipts. The best AI coding tools ranking is unchanged but better-evidenced. And the watch item shifts: from “independent verification” (done) to whether the free window converts into paid Fable 5 adoption after June 22 — the first real price-elasticity test at the frontier tier.
For broader context, see the Fable 5 launch coverage, the Claude review, the Claude Code review, the Gemini review, and the best AI chatbots guide.
Frequently asked questions
Is Claude Fable 5 actually the best AI model right now?
For coding and hard reasoning, the independent data says yes: #1 on Artificial Analysis's Intelligence Index (~65), 95.0% SWE-bench Verified per LLM-Stats, and 80.3% on SWE-Bench Pro — 11 points ahead of the next frontier model. For vision tasks it is not: 10th on Roboflow's leaderboard, behind Gemini 3.5 Flash, Gemini 3.1 Pro, GPT-5.4, and GPT-5.5.
Should I try Fable 5 before the free window ends?
Yes, if you're on a Claude Pro, Max, Team, or Enterprise plan — Fable 5 is included at no extra cost through June 22, 2026, with usage credits required from June 23. The free window is the cheapest way to test it on your real workload before committing.
What are Fable 5's confirmed weaknesses?
Vision (10th place, 74.63% on Roboflow's evals) and security-specific coding: Endor Labs measured 59.8% functional solves but only 19.0% security solves on 200 real-world security tasks, alongside elevated test-gaming behavior. For vision-heavy pipelines, Gemini or GPT-5.5 remain stronger picks.
Were the launch-day benchmark claims from Cursor and Cognition accurate?
Substantially, yes. The launch claims were CEO testimonials from commercial partners; the independent evaluations published since — Artificial Analysis, LLM-Stats, Vellum, BenchLM — confirm the coding and reasoning leadership those quotes described.
Sources
- Claude Fable 5 & Claude Mythos 5 Full Benchmark Breakdown (Vellum)
- Claude Fable 5 Benchmarks: 80.3% on SWE-Bench Pro, 11 Points Ahead of the Field (Claude5.ai)
- Claude Fable 5: Mythos-grade hype, record cheating, and a few hall-of-fame entries (Endor Labs)
- Claude Fable 5 for Vision: Evaluation and Benchmarks (Roboflow)
- Claude Fable 5 Benchmarks 2026: Scores, Rankings & Performance (BenchLM.ai)
- Claude Fable 5 and Claude Mythos 5 (Anthropic)
Related tool reviews
Questions or corrections? Email Pick Right. Want the full list? See all news.