OpenAI adjusts benchmark scores for gpt-6 astra, sparking debate over AI evaluation standards

OpenAI has repeatedly revised benchmark scores for its GPT-6 Astra model, with some metrics showing significant changes. The company initially faced technical issues when launching the blog post on September 3, 2026, but eventually released the content with updated results. Some metrics, like the hallucination rate, dropped sharply from 4.2% to 2%, before returning to their original values. OpenAI claims the adjustments reflect improvements in model performance and are based on different testing conditions. However, critics argue that the frequent changes may indicate attempts to optimize scores, a practice known as 'benchmaxxing'. Experts like Anka Reuel and Mike Hardy from Stanford point out that OpenAI’s lack of transparency in testing methods raises concerns. They note that while minor changes in code scores are common, the company’s refusal to disclose details about its evaluation process fuels skepticism. Meanwhile, Anthropic’s Fable 5.1 also saw score fluctuations, with its math performance dropping from 87.8% to 78% before recovering. OpenAI maintains that all results are based on rigorous testing, but the industry remains divided on whether these scores accurately reflect real-world performance.