Experts weigh in on AI risks: nuclear threats and governance concerns dominate safety debate

A survey of ten AI safety experts revealed widespread concern over potential dangers, with few adopting an overly optimistic view. Key risks highlighted include the integration of error-prone AI into nuclear operations, which one expert deemed incredibly urgent, and the potential for AI-enabled biological attacks, which could drastically reduce the timeline for producing pathogens. Panelists expressed skepticism regarding a pause in AI development, noting that major corporations are closely tied to capitalist structures. While some sources felt governments would be too slow to regulate, computer scientist Yoshua Bengio urged the public to pressure politicians. Concerns also centered on the concentration of power in the US-China race and the danger of 'democratizing AI' without corresponding governance. Specific risks cited included cybersecurity vulnerabilities, with one expert noting infrastructure has not adopted new defenses in recent years, and the danger of lab leaks, where AI assistants could lower barriers for dangerous experiments. Conversely, one safety engineer dismissed the notion of human extinction as an overblown risk, insisting that claims must be falsifiable and verifiable, contrasting this with the rigorous standards developed during the Manhattan Project.