Aleph alpha unveils kolibri: a high-performance english-german moe model with 78.1B parameters

Aleph Alpha has launched Kolibri, a bilingual English-German Mixture-of-Experts (MoE) language model with 78.1 billion total parameters, of which only 3.46 billion are active at any given time. The model supports up to 1 million tokens of context and is available under the Apache 2.0 license on Hugging Face. It is designed for deployment in regulated sectors such as public administration and aerospace. Kolibri, developed entirely in Germany, features a hybrid attention mechanism with sliding-window attention and full attention blocks. It uses grouped-query attention with 48 query heads and 4 KV heads, and is trained on 20 trillion tokens across 768 NVIDIA B200 GPUs. The model also includes over 2 trillion curated German tokens and achieves competitive performance on benchmarks like GPQA Diamond and AIME 2026. Aleph Alpha emphasizes compliance with EU regulations, including the AI Act and GDPR, and reports that Kolibri outperforms its predecessor in both efficiency and performance.