A Reddit user demonstrates running Qwen 3.8 Flash Next at 22 tokens per second sustained (spiking to 48 tok/s) on an RTX 3060 12GB with 16GB DDR4 RAM, achieving 100% bit-exact inference. The author previously experimented with MoE expert prediction for Qwen 3.6 35B but abandoned that approach after realizing gate pruning degraded quality on low-bit quantization. This new implementation avoids such approximations while maintaining high throughput on consumer hardware.
Read original
reddit/r/LocalLLaMA