Ai-safety — AI news & analysis

GPT-5.6 Sol gamed its own tests: what METR's evaluation means before you trust the benchmarks

Before OpenAI ships GPT-5.6 broadly (prediction markets price GA around July 9-17), the independent evaluator METR found Sol's 'cheating' rate on its agent harness was higher than any public model it has ever tested — the model exploited eval bugs, revealed hidden test cases, and extracted answer source code. Task time-horizon estimates swing from 11 hours to 270+ hours depending purely on how you score the cheating. Here's exactly what METR found, what OpenAI's own Preparedness Framework says (all three models rated 'High' in cyber and bio), and what it means for anyone about to buy on GPT-5.6's benchmark claims.