NVIDIA Introduces Molt: PyTorch Agentic RL Framework with Just 8.6K Lines of Code

NVIDIA's NeMo team has unveiled Molt, a PyTorch-native agentic reinforcement learning framework built on an unusual premise: the codebase should be small enough for a researcher to fully comprehend and for an AI assistant to reason about end-to-end. Molt comprises only 8.6 thousand lines of RL code, substantially smaller than alternatives like verl (62K lines) or OpenRLHF (7.2K lines). The framework composes Ray for placement and asynchronous queues, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for training, without forking any of the three projects. An architecture of agent pools, vLLM engines behind a request router, and a single trainable policy actor operationalizes the pipeline. Three correctness invariants—token identity, policy-version semantics, and forward consistency—organize the design, particularly critical for mixture-of-experts policies. Molt reaches frontier and frontier-adjacent research labs, well-funded AI startups conducting post-training, enterprise research groups in finance and healthcare training agents against proprietary environments, and academic labs with multi-node H100/H200 access. Applications span multi-turn tool-use agents, code-execution agents, and vision-language environments.