Open AI research initiative seeks transparent development of high-risk AI technologies

Nathan Lambert and Tom Zick have launched Trillium Labs, a nonprofit focused on transparent, high-stakes AI research. The initiative aims to address risks like recursive self-improvement (RSI) and reinforcement learning by making experimental details publicly available for scrutiny and replication. Lambert argues that the current closed approach to AI development hinders risk assessment, advocating for a return to the scientific method. Zick emphasizes the importance of publishing insights from reinforcement learning experiments to reveal AI behavior. The nonprofit has raised undisclosed funding from Schmidt Sciences, Halcyon Futures, and others, with plans to raise $40 to $100 million in total. Tim Fist of the Institute for Progress supports greater transparency in AI R&D, highlighting its potential to enrich broader discourse on AI development. Lambert and Zick met during the pandemic while studying at UC Berkeley and were inspired to start Trillium Labs after observing the growing disconnect between academic research and industry practices. The nonprofit will initially focus on post-training fine-tuning and RSI, areas of concern due to their potential for uncontrolled AI advancement. Recent warnings from an Anthropic researcher about RSI’s existential risks have intensified the debate over open versus closed AI development. The initiative seeks to contribute a more nuanced perspective to the ongoing discussion about how to responsibly build AI systems.