Inside the labs: AI researchers sound alarm over accelerated development

AI researchers within major labs are increasingly warning about the rapid development of advanced AI systems, signaling a shift from external criticism to internal concern. Jacob Coxon, a former Anthropic researcher, resigned and publicly warned that labs are racing toward self-improving superintelligence, risking human safety. His warning resonated with other Anthropic researchers, including Evan Hubinger, Ethan Perez, and Samuel Marks, as well as OpenAI’s Julie Steele and Jasmine Wang. Coxon highlighted the concept of recursive self-improvement, where AI models are used to design and optimize future models, effectively taking over parts of the development process. OpenAI recently reported that its coding agents are accelerating research, with 3.1 agent-workdays per human workday by mid-August. Anthropic and Google have also noted that frontier AI models are contributing to the development of their successors. These tools, while boosting productivity for enterprises, also enable faster AI model development within labs.