Alibaba unveils Qwen3.8-Flash-Next: a 125b multimodal moe model with 6b active parameters

Alibaba’s Qwen team has introduced Qwen3.8-Flash-Next, a new open-weight multimodal Mixture-of-Experts (MoE) model designed for cost efficiency per token. The model combines a 125B backbone with a 51B N-gram embedding table and a 3B multi-token prediction module, totaling 180B parameters on disk. Only 6B parameters are activated per token, marking a significant reduction in computational demand. This release is positioned as an early preview of the architecture that will form the basis of Qwen4, similar to how Qwen3-Next served as a precursor to Qwen3.5. The model incorporates several key innovations, including a Gated DeltaNet and Qwen Sparse Attention hybrid, Gated Residual, N-gram Embedding, and the Muon optimizer. Training costs are reported to be roughly one-ninth of those for Qwen3.7-Plus. The FP8 checkpoint is 172.78 GiB, while the BF16 version is 335.28 GiB. According to vLLM recipes, TP2 is the minimum validated FP8 configuration on GB300, with TP4 recommended. On an 8×H200 node, TEP8 is advised, as plain TP8 is incompatible with the checkpoint’s 128-wide quantization blocks. Sparse activation reduces compute requirements without affecting storage. Qwen3.8-Flash-Next achieves strong performance across various benchmarks, including 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 81.0 on SWE-bench Multilingual, and 91.9 on LiveCodeBench v6. In agentic tasks, it scores 73.9 on CoWorkBench, 55.7 on JobBench, and 73.5 on Toolathlon Verified. Multimodal results include 84.5 on AndroidWorld, 76.6 on LVBench, 88.5 on RealWorldQA, and 95.7 on MathVision with code interpreter. However, it does not lead in all areas, with Claude Opus 4.6 (Max) achieving 40.0 in HLE compared to Qwen’s 35.9, and DeepSeek-V4-Flash-0731 leading NL2Repo-Bench at 54.2 versus 48.1. Frontier reasoning remains a challenge. The model supports serving through vLLM, SGLang, TokenSpeed, transformers serve, and llama.cpp for GGUF quantization. Fine-tuning is available via Unsloth, Swift, and LLaMA-Factory. It already powers the 'Standard' mode on QwenWork and integrates with Qwen Code. The model defaults to thinking mode with reasoning_effort set to xhigh, medium, or low. Qwen recommends temperature 1.0 and top_p 0.95 for thinking mode, and temperature 0.7 with top_p 0.80 for instruct mode. Context length is 262,144 tokens natively, extendable to 1,000,000 with YaRN. The model reports up to 7.6× prefill and 4.9× decode speedups with the QSA kernel, while the SGLang cookbook and vLLM recipes cite 10.2× and 6.6× respectively. Qwen also claims an 8.6× increase in prefill throughput over Qwen3.7-Plus at a 90% prefix-cache hit rate.