ML engineer Alish from Nace introduces KQuant, a new quantization technique achieving 3.7× better quantization for Qwen models (Flash Next 180B, 27B, 35B, 9B) measured by KL-divergence reduction. Developed with a University of Toronto professor using Fable 5.2, it establishes a Pareto frontier for accuracy-size trade-offs in the Qwen family, initially for MLX with GGUF support planned.
Read original
reddit/r/LocalLLaMA