Vixhal introduces qwenGPU: a model integrating cuda execution directly into weights

Developer Vixhal has released qwenGPU, an experimental model that embeds a SIMT GPU directly within the weights of Qwen3.5-2B. This implementation allows the model's forward passes to execute real CUDA, compiled into NVIDIA PTX, testing the concept of computation occurring inside a pre-trained LLM rather than alongside it. The GPU component consists of 45 un-trained transformer layers placed before Qwen’s native 24 layers, utilizing Qwen3.5’s Gated DeltaNet and SwiGLU MLPs. This integrated hardware simulates a 4-lane SIMT machine featuring 15 registers per lane and supporting features like atomicAdd and divergent branches. The host system handles the compilation of CUDA to PTX using clang, feeding the program and inputs as tokens to the model. The model then generates execution tokens, such as EXEC(b, w), which trigger block execution, with results returned via MEM records. While the model's next-token distribution remains very close to the original, with a KL divergence of approximately 7e-4, the implementation is complex. The quickstart requires Python 3.11+, uv, just, and about 16 GB of memory, and advanced mode necessitates clang with the NVPTX backend. The project utilizes the transformer compiler from Qwenputer by Divyansh Agrawal, which previously compiled a 32-bit RISC-V CPU into Qwen3.5-4B.