NVIDIA enhances kimi k3 model inference speed with vllm optimization

On October 18, 2026, SemiAnalysis reported that NVIDIA improved the inference speed of the 2.8 trillion parameter Kimi K3 model by 1.68 times using an upstream vLLM optimization. The model now achieves 423 tokens per second per user on the same hardware (GB300 NVL72). The optimization was developed by NVIDIA CentML engineer Benjamin, who implemented large-scale Megakernel fusion using Cute DSL to write a decoding-specific kernel. The Megakernel, consisting of 12,000 lines of code, represents a direction for advanced AI-assisted computation kernels. NVIDIA is also working on optimizing other models like DeepSeek V4.1 for vLLM.