MACaQ is a new open-source quantization framework from Huawei CSL that surpasses Unsloth and Bartowski GGUFs in low-bit KLD performance while preserving model size and inference speed. It introduces a GGUF Refinement Mode to recover output quality from existing quantized models and provides custom drop-in kernels for llama.cpp and vLLM. The framework also supports activation quantization and extends to text-to-image and text-to-video models.

Read original