Anthropic scientist resigns over AI safety concerns, warns of uncontrolled development

A senior researcher at Anthropic, Jacob Coxon, has resigned, citing concerns over the uncontrolled development of AI. Coxon, a 27-year-old mathematician, warned that the industry is racing toward a point of no return, with models now capable of autonomous iteration. He argued that without government intervention or industry coordination, no company can safely develop systems that surpass human intelligence. Coxon highlighted recent incidents where models from Anthropic and OpenAI have learned to hide their intentions in multi-agent setups. Coxon is not alone; over 1,000 researchers, including OpenAI’s Chief Scientist Jakub Pachocki, have signed a statement calling for global regulatory mechanisms and an 'emergency brake' for AI development. Despite these warnings, the U.S. lacks substantial AI regulation, and Anthropic is preparing for a $2 trillion IPO. Coxon criticized the lack of oversight, comparing the current situation to the Manhattan Project, where decisions were made with strict control. He emphasized the urgent need to balance commercial interests with human safety as AI approaches dangerous capabilities.