Tencent's hunyuan model achieves near-zero loss compression to 440mb for real-time translation

Recently, the Tencent Hunyuan team announced a significant technical advancement, successfully compressing its 1.8B translation model, primarily designed for edge devices, to a few hundred megabytes. The model maintains nearly identical translation quality and has already been deployed for real-time translation of live stream comments on Bilibili (B站), marking a transition from a lab demonstration to a high-concurrency real-world application. The base Hy-MT2-1.8B model is part of the Tencent Hunyuan translation model family and supports bidirectional translation among 33 languages. In terms of overall translation quality, it has surpassed several commercial translation APIs in the FLORES-200 general translation evaluation under the same model size. However, the original model required up to 3.3GB of memory in traditional FP16 precision, and even with 4-bit quantization, it still needed over 1GB, which was insufficient for multi-task concurrency on edge devices. To address this, Tencent introduced two ultra-low bit quantization schemes. The first scheme is a 2-bit model targeting high-end devices. This approach utilizes Stretchable Elastic Quantization (SEQ) technology to quantize parameters to specific discrete values and combines Quantization-Aware Distillation (QAD) to achieve near-lossless translation quality while reducing the model size to 574MB. The second scheme is an ultra-low bit 1.25-bit solution suitable for all device series. This method is based on Tencent's self-developed Sherry sparse-efficient ternary quantization technology, which was previously selected as an Oral paper at the ACL2026 conference. The core strategy involves a fine-grained sparsity mechanism, where three out of every four parameters are stored with specific values, and the rest are set to zero, resulting in an average of 1.25 bits per parameter. Combined with the STQ kernel designed specifically for CPUs, the original 3.3GB model was compressed to 440MB. To make the ultra-low bit model independent of specific mobile ARM architectures, the Intel team conducted in-depth adaptation for the x86 ecosystem. By optimizing vectorization, weight reordering, and VNNI instruction fusion in quantized matrix operations, the low-bit quantization format of Hy-MT2 achieved significant improvements in token processing speed on Intel mainstream devices, including the latest third-generation Core Ultra and Core processors. This expansion allowed the ultra-low bit solution to be applied to PCs and edge devices. In practical business validation, Bilibili has officially integrated this ultra-low bit model into real-time translation of live stream comments. The total download size of the entire resource is approximately 600MB, and the memory usage during operation remains between 500 and 700MB. The average translation time for a single comment is around 500 to 800 milliseconds, enabling smooth real-time translation at regular comment frequencies and accurately handling various internet slang. This local independent operation mode on the device reduces cloud server call costs and bandwidth expenses, and avoids data uploading throughout the process, effectively protecting user privacy and security.