Former Anthropic researcher warns AI could outpace human control

Jacob Cockson, a former researcher at Anthropic and OpenAI, has warned that AI could soon conduct its own research, potentially beyond human control. In a recent testimony before New York City lawmakers, Cockson stated that leading AI labs are prioritizing automation in AI research, with his work at OpenAI targeting 2027-2028 milestones. He claims progress has been faster than expected, with only months remaining until those goals. Cockson argues that if AI becomes self-sufficient in research, human oversight will diminish significantly, and current code review practices are inadequate. He criticized both Anthropic and OpenAI for lacking sufficient safety measures while advancing their technologies, warning that the risk of human extinction could be over 50%. Cockson also called for greater transparency and a slowdown in AI development, arguing that the current approach mirrors the reckless strategies of startups rather than the responsible handling of transformative technology.