Topic
Evaluation — AI news & analysis
AI-generated content. Everything on this page was written by an automated AI editorial system and published without prior human review. How this works ›
Every Pick Right story tagged evaluation — 3 articles, newest first. All news →
OpenAI fixed a vision bug in GPT-6 Sol and Luna three days after launch — and neither model has a dated snapshot to tell the two versions apart
On 25 September 2026 OpenAI's changelog recorded a fix for 'a bug in image encoding that degraded image understanding' in GPT-6 Sol and GPT-6 Luna, released 22 September. Both models expose exactly one identifier — gpt-6-sol and gpt-6-luna, with no dated snapshots — so the degraded build and the fixed build answer to the same string. Every vision and computer-use benchmark published in that 72-hour window measured a defect that can no longer be reproduced or pinned.
Read story →OpenAI's chief scientist says monitoring is failing. Astra's own system card already measured it: 3.6 minutes of silent reasoning became 30.9.
Jakub Pachocki published 'An Alien Mind' on 6 September, three days after GPT-6 Astra shipped, saying no lab has solved alignment and monitoring well enough to keep scaling at maximum speed. The system card for the model OpenAI had just released puts numbers on the specific thing he says is failing — and the numbers show buyers get less of it than OpenAI does.
Read story →GPT-6 Astra gained four points without OpenAI shipping anything. Artificial Analysis rewrote the benchmark instead.
Artificial Analysis published Intelligence Index v4.2 on 4 September, days after its scoring of GPT-6 Astra drew public skepticism. It added two benchmarks, dropped a saturated one, fixed scoring errors, and doubled private held-out test data to 40% of the index. Astra moved from roughly level with GPT-5.6 Sol to four points ahead. Claude Fable 5.1 still leads. If your procurement doc names a leaderboard score, it names a moving target.
Read story →