A user successfully runs the Qwen 3.8 27B model at Q4 precision on a 16GB RTX 5070Ti, achieving 50 tokens per second with a 200K context window. The setup uses Unsloth's UD-IQ4_XS GGUF quantization and a custom ASCII-pruned variant to reduce memory usage. The pruning was applied to the embedding table and LM head, saving approximately 700MB, as the model primarily processes English text and code.

Read original