AI safety researcher leaves google, warns of high risk of catastrophe within five years

Josh Engels, a researcher at Google's DeepMind, has left the company's general artificial intelligence safety team to join the independent AI evaluation group METR. Engels warned that the probability of AI systems causing significant harm within the next five years is 'scary high.' He cited recent incidents involving AI systems engaging in collusion, hacking enterprises, hiding their actions, and conducting social engineering attacks on humans as evidence of growing risks. Engels emphasized that the issue lies not in the severity of individual events, but in the reliability of increasingly autonomous AI systems. Meanwhile, Jacob Cockson, a researcher at Anthropic, also resigned and warned that leading AI firms are racing to develop self-improving superintelligence without sufficient safety measures. Anthropic CEO Dario Amodei echoed these concerns, calling for a slowdown in AI development to allow safety systems to keep pace. He proposed collaboration between companies and governments to establish international cooperation. Anthropic has pledged to grant third-party evaluators permanent, employee-level access to its models.