The author runs the IQ3_XXS quantized version of Qwen3.8-Flash-Next (125B MoE) at its full 262,144‑token context length on a single RTX 3090 with 32 GB RAM, using the Strata inference engine. Performance metrics show ~1,325 tokens/s during prefill, 59‑62 tokens/s decode at long context, and 63‑100 tokens/s decode at short context, with no data read from NVMe storage. They seek community feedback on whether these results are typical or can be improved.
Read original
reddit/r/LocalLLM