Inkling: Open-Weight Model at Nearly 1 Trillion Parameters with Novel Architecture

Thinking Machines Lab unveiled Inkling, an open-source language model containing 975 billion parameters structured as a sparse Mixture-of-Experts system. Only 41 billion parameters are active during inference, and the model supports context windows reaching one million tokens. Benchmark comparisons against GLM-5.2 reveal mixed results—Inkling outperforms on certain metrics such as 79.8% on IFBench versus 73.3% from the competitor, but lags on reasoning and coding tasks. The lab positions Inkling as a versatile foundation model designed for fine-tuning and downstream specialization, complementing their Tinker customization platform. The architecture incorporates several unconventional technical choices: short convolutional layers embedded across attention mechanisms, an additional RMSNorm normalization following token embeddings, and learned relative position biases instead of the prevalent RoPE approach. Within this design, 55 of the 66 decoder layers employ local attention with restricted context windows, a configuration that works synergistically with the relative position methodology.