The author benchmarked all Qwen 3.8‑27B unslother GGUF quantizations that fit within 16 GB VRAM using llama.cpp’s llama‑perplexity on the wikitext‑2 dataset (≈280k tokens, n_ctx = 512, fp16 KV cache). Perplexity scores, which are deterministic forward passes, were compared across quant variants having the lowest scores indicating models closest to the full‑precision baseline. The results provide a reproducible metric for selecting the best quantization for limited‑VRAM deployment.

Read original