Princeton AI lab's sson benchmark reveals significant gap between human and AI 3d spatial reasoning

The Princeton University AI Lab has introduced the SSoN benchmark, a new test designed to isolate and evaluate an AI's ability to understand 3D space under static, cross-view conditions, removing the influence of dynamic interaction. Testing involved asking models to identify the odd image among four static indoor scenes. In this task, human participants achieved an accuracy of 83.3%, whereas the best mainstream vision-language models scored only 39.7%, with most models performing worse than random chance, highlighting a fundamental weakness in current AI's geometric reasoning. Analysis of model failures pointed to three issues: high error rates in reasoning involving spatial transformations like rotation, instances where models provided logically invalid explanations for correct answers (nearly 30% of cases), and overall performance below random guessing. Intervention experiments showed that while providing real camera parameters improved performance, targeted supervised fine-tuning was most effective, allowing models to nearly double their accuracy on unseen rooms, suggesting the gap is addressable through focused training.