Adaptive KV streaming and RDNA 4/ROCm optimizations in llama.cpp enable Qwen 3.8‑27B (IQ4_XS weights, Q8_0/Q4_0 KV cache) to run its full 262,144‑token context on a single 16 GB RX 9070 XT GPU, achieving >80 tokens/s decode. The ASCII‑condensed version reduces the model to ≈13.6 GB (12.6 GiB), leaving ~3.4 GiB of usable VRAM for the KV cache and overhead.
Read original
reddit/r/LocalLLM