Polymathic AI and princeton team introduce FlashMap, a new tool for accelerating LLM decoding

Researchers from Polymathic AI, Princeton University, USC ISI, NYU, and the Flatiron Institute have unveiled FlashMap, a novel speculative decoding drafter that significantly enhances the speed of large language model (LLM) decoding. FlashMap replaces traditional discrete diffusion methods with a continuous flow map, enabling the generation of a full 16-token block in a single forward pass. This method ensures token consistency within the block and achieves up to 6.14x speedup on the Qwen3-8B model for GSM8K tasks. The technique leverages a 6-block diffusion transformer (DiT) with an embedding dimension of 2048 and 16 attention heads. It reads hidden states from specific target layers and copies the embedding matrix and LM head from the target model, keeping them frozen during training. Training involved two stages using 500K samples from Open-PerfectBlend, with total compute reaching approximately 4,600 GPU hours on H200s. The project also provides installation instructions and serving options, including vLLM and SGLang plugins, for deploying FlashMap efficiently.