The author shows that a single NVIDIA RTX 9060 XT with 16 GB VRAM and 24 GB DDR5 RAM suffices to run Qwen 3.8 27B at a 262 k token context length. Using unsloth IQ4XS ASCII‑condensed quantization, KV‑cache streaming with Q4 values and Q8 keys, MTP+ n‑gram prediction, they achieve ~42.7 tokens/s generation and 658 tokens/s prefill at 20 k context. This demonstrates that expensive, high‑end builds are not required for capable local LLM inference.
Read original
reddit/r/LocalLLaMA