Perplexity and NVIDIA unveil jolt: a new reinforcement learning technique for language models

Perplexity and NVIDIA have introduced JOLT, a novel post-training method for language models that combines teacher and student roles within a single model. JOLT, standing for Joint On-policy Learning and Teaching, enables the model to act as both a teacher with privileged information and a student without such access. This approach addresses limitations of traditional reinforcement learning methods like GRPO, which suffer from sparse reward signals and slow learning on complex tasks. JOLT improves efficiency by reducing the number of training tokens needed, achieving 6.5x fewer processed tokens than GRPO on the MATH benchmark. The method also outperforms GR,PO in several benchmarks, including GSM8K, LCBv6, and AppWorld, with significant accuracy gains. The research team tested JOLT across five settings, with JOLT+ achieving the highest scores in all cases. The study demonstrates that JOLT’s reward-consistent teacher ensures alignment with the student’s learning objectives, leading to better performance and faster convergence. Perplexity’s research team reported that even the basic version of JOLT outperformed GRPO on MATH, achieving 72.14% accuracy compared to 68.36%. The method also shows a higher plateau in accuracy, reaching near 72% versus 70% for GRPO and 62% for OPSD. The findings, detailed in the paper on arXiv, underscore JOLT’s significance in advancing post-training techniques for large language models.