Opti 27B compresses Qwen3.8-27B (text+vision) to an 11.8 GB GGUF file at 3.47 bits/parameter, achieving a Wikitext‑2 perplexity of 6.487 (within 0.5% of the FP16 original) and matching Q4_K_M performance while using 30% fewer bytes. A patched llama.cpp build enables ~42 tok/s on an RTX 3090 and fits four 16k vision conversations in 17 GB, suitable for a 16 GB GPU. Source code and model are publicly available.

Read original