PrismML has released Bonsai 2 27B, a highly compressed model based on Alibaba's Qwen3.8-27B that achieves near-lossless performance at a fraction of the original size. By replacing most weights with ternary values (-1, 0, +1) and utilizing a Hadamard rotation to distribute outliers, the model reaches an effective 1.72 bits per weight. This methodology allows the 54GB FP16 baseline to be compressed into a 5.9GB file while retaining 98.2% of its benchmark average. Notably, the model maintains or even exceeds baseline performance in complex reasoning tasks, scoring 95.83 on AIME26 and 90.07 on LiveCodeBench, whereas conventional 2-bit quantization (IQ2_XXS) significantly degrades in these areas. Hardware performance is substantial; an Apple M5 Pro achieves approximately 28 tokens per second, making a 27B-class reasoning model interactive on consumer laptops. While the model excels in math and coding, it shows performance drops in knowledge retrieval, vision, and OCR. To run the model, users must use PrismML's specific llama.cpp fork to support the necessary ternary kernels and Hadamard activation runtime, as standard implementations will fail to process the weights correctly.
Read original
dev.to