AI safety initiatives may grant major labs a strategic edge

Investors and analysts suggest that major AI labs, particularly Anthropic, are leveraging the current AI safety concerns to push for regulations that only large labs can easily meet. These regulations could financially disadvantage smaller labs and open-source developers. PitchBook analyst Harrison Rolfes notes that recent safety announcements from big labs may appear performative, akin to AI safety 'theater.' Critics argue that such regulatory capture could limit the number of companies offering AI to a few major players like OpenAI, Anthropic, SpaceXAI, Google, Microsoft, and Mistral. Independent evaluation firms, such as METR, Redwood Research, and Apollo Research, might act as regulators, similar to the Big Four accounting firms. Anthropic CEO Dario Amodei proposed embedding independent safety evaluators within AI companies, a stance supported by OpenAI CEO Sam Altman. Rolfes warns that these evaluators could create a 'moat' around major labs, as smaller labs may struggle with the costs of third-party safety assessments. Apollo Research's Marius Hobbhahn highlighted that evaluation costs rise with model complexity, as models now act as LM agents, requiring increasingly complex tests.