Local AI lab boosts Qwen3.6 speed 6x on one RTX 4090

An open-source project called Lightreason achieved a significant performance leap for running large language models locally. By combining existing techniques like NVIDIA CUDA Graphs and novel optimizations for weight management, it sped up a 35-billion parameter Qwen model from about 8 tokens per second to over 51 tokens per second on a single RTX 4090 GPU, a 6.4x improvement.

For developers and researchers running models locally, this means vastly faster iteration and lower hardware costs for experimentation. The techniques focus on reusing prepared GPU work, checking multiple tokens together, and intelligently swapping model weights between GPU and system RAM. The takeaway is that clever software optimization can dramatically extend the usable limits of consumer hardware, making powerful AI more accessible.