Kimi K3: A 2.8 Trillion-Parameter Open-Source Model Built for Inference Efficiency

Kimi released Kimi K3, an open-source language model with 2.8 trillion parameters — currently the largest open-weight model in existence. The architecture represents a major scaling expansion of Kimi Linear, which debuted the previous year at 48 billion parameters. The model prioritizes inference efficiency through multiple architectural optimizations. Traditional Mixture of Experts has been replaced with LatentMoE, standard attention mechanisms with multi-head latent attention, and introduces Kimi Delta Attention. The design incorporates attention residuals to strengthen the residual pathway by connecting residuals across layers using attention-based importance weighting. A distinctive choice is the complete removal of rotary positional embeddings (RoPE) in favor of NoPE applied uniformly across all layers — making it the first frontier-level model to adopt this strategy globally. The model also provides native multimodal support, broadening its capabilities beyond text.