Shanghai AI lab and shanghai jiao tong university introduce groundbreaking ncp model

On September 22, 2026, the Shanghai Artificial Intelligence Lab and the School of Artificial Intelligence at Shanghai Jiao Tong University unveiled a new pre-training paradigm for Next Concept Prediction (NCP). Their model, NCP-ArchPreview, features an 8.9B-scale discrete latent space and matches the performance of OLMo-3-7B. The model was pre-trained on 5.73T of Dolma-3 data, using only 51.3% of the token budget to achieve comparable results, improving convergence speed by 1.95 times and computational efficiency by 1.74 times. The NCP-ArchPreview employs a three-stage latent space processing pipeline, utilizing product quantization to build a large concept representation space. It predicts future concepts through differentiable next concept prediction and causal leakage prevention mechanisms, shifting from word-by-word prediction to latent space concept prediction. The model also demonstrated superior performance in downstream tasks, outperforming by 2.45 points and achieving a 6-point improvement in GSM8K mathematical reasoning. The team also found that fine-tuning just 17M latent space parameters surpassed LoRA, without forgetting issues. Speculative decoding acceptance length increased by 4.17%, reaching 7.59% in code tasks. The researchers shared their experiences, built a trillion-scale proxy metric screening mechanism, and open-sourced all assets, including full-stage checkpoints and evaluation frameworks.