The rise of world models: how AI is learning to simulate reality

The technology underpinning world models, which allows AI to generate simulated environments based on visual input, is rapidly advancing. Niantic, the creator of Pokémon Go, is focusing on reconstructing the real world using its vast data trove, as stated by Brian McClendon, Niantic Spatial’s chief technology officer. Unlike language models that predict the next word, world models predict the next visual state from an image, enabling the creation of realistic digital training grounds. These simulations are set to impact fields like home robotics and autonomous vehicles, with 2027 noted as a potential turning point for expansion. Jamie Shotton, chief scientist at Wayve, emphasized that for self-driving cars, a world model must grasp physics and cause-and-effect to react to real-world actions. Wayve utilizes dashcam footage to build models that recreate hazardous road conditions for testing. Beyond physical applications, the technology is attracting significant investment; Yann LeCun, Meta’s former chief AI scientist, recently secured over a billion dollars for his Advanced Machines Intelligence startup. Furthermore, Fei-Fei Li launched World Labs in 2024, which AMD acquired this year for over $8 billion. The need for data collection to train these models is also creating new forms of gig work, as illustrated by the author's experience recording personal activities for data sale.