AI engineers test language models' driving skills with toyota corolla

Three AI engineers at Axiom, Aditya Ramabadran, Simon Mahns, and Tobias Gessler, used OpenAI’s GPT-6 Astra to drive a 2024 Toyota Corolla to an In-N-Out Burger in the Bay Area. The AI model, normally designed for text and code generation, navigated the car to the drive-thru window using cameras and power steering, with a safety driver monitoring. The experiment highlights the growing physical understanding of language-based AI models. Researchers argue that while current models excel at virtual tasks, their real-world capabilities remain limited. Andrew Dai of Elorian AI emphasized the importance of physical reasoning for robotics and home automation. A new benchmark, Humanity’s Sixth Sense, evaluates models’ ability to understand physical scenes. The engineers also developed DrivingBench, which shows models like Astra can complete a simple parking lot course, though slowly. Despite progress, models still struggle with complex driving tasks. Ramabadran noted that models adapt in real-time, suggesting emergent capabilities from multimodal training. However, the trio’s experiment underscores the gap between virtual and real-world AI performance.