Baidu releases ernie 5.5 flash base model with 280b parameters, offering open weights

Baidu's Wenxin AI has publicly released the ERNIE-5.5-Flash-Base model, making the weights of its pre-trained endpoint available. This sparse Mixture-of-Experts (MoE) language model boasts a total of 280 billion parameters, activating only 14 billion per token, and was pre-trained from scratch on 33.5 trillion tokens. While the base model is provided as a starting point for research, Baidu stated that a subsequently fine-tuned version, ERNIE 5.5 Flash, will follow. The model's training utilized the Muon optimizer and FP8 mixed precision, with a maximum training context of 128K tokens, although inference and fine-tuning support up to 1M tokens. The training corpus emphasized three areas: code data, which included software evolution data; long-form data, featuring natural long documents and synthetic QA; and specialized data curation. Technically, the system design featured a distributed Muon optimizer that merges cross-machine communication for Dense and MoE parameters, reducing latency. ERNIE-5.5-Flash-Base was evaluated across 30 benchmarks covering seven capabilities, including code and long context, and its performance was compared against DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base.