A user successfully ran the Qwen3.8-27B dense model fully on a single RTX 5060 Ti 16GB GPU, achieving 45–47 tok/s. The setup utilizes a custom Unsloth IQ4_XS quantization via llama.cpp with a 32K context window and MTP enabled. Performance is optimized through Flash Attention, CUDA Graphs, and Q4 KV caching.

Read original