Four AI leaders unify efforts to establish global standard for content opt-outs

OpenAI, Google, Microsoft, and Anthropic have collaboratively agreed to implement a new industry standard allowing website owners to issue a singular declaration refusing AI crawling. This unified mechanism, submitted to the Internet Engineering Task Force (IETF) by Anthropic's technical lead, Kevin Kelley, will be integrated directly into existing robots.txt files. This extension updates the function of robots.txt beyond merely controlling search engine access to governing AI training data usage. Previously, publishers needed to negotiate their content restrictions with each AI company individually; the new standard streamlines this into one notification. Anthropic has already adopted the standard. OpenAI plans to implement it within the coming months, while Google and Microsoft are anticipated to follow next year. Furthermore, a spokesperson for Meta voiced support for establishing a common signal, although a firm timeline for their adoption remains unset. This agreement among the four most powerful AI entities marks a significant shift toward structured dialogue regarding content rights.