A unified framework for building world models across multiple domains

Researchers from PhAI Labs, CUHK, Fudan, Stanford, Oxford, and Princeton have introduced JEPA-Anything, a domain-agnostic framework for constructing world models. This approach uses a single learning recipe to build predictive models across seven distinct fields, including vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. The framework extends joint-embedding predictive architectures (JEPAs) with a method called Orthogonal Predictive Factorization (OPF), which splits latent targets into subspaces to improve model stability and performance. Key results include significant improvements in single-cell data clustering, clinical event forecasting, and molecular dynamics simulations. On the UK Biobank dataset, JEPA-Anything achieved a mean PRAUC of 0.718, surpassing the standard JEPA's 0.711. In molecular rollouts, it demonstrated the lowest MAE and RMSD for water, quartz, paracetamol, and benzene. Additionally, the framework identified potential cancer interventions, with wet-lab tests supporting findings on IL-18 and CD73 blockade. While planning results were mixed, JEPA-Anything showed improvements in some environments like Walker2d and HalfCheetah. The research team also noted that latent orbital modes recovered Kepler’s law with a fitted slope of −1.4991, closely matching the theoretical −1.5. The project's GitHub repository and model checkpoints are available for further exploration.