Independent researcher Aman Sachan releases MAPLE (Manifold-Activated Pruning of Latent Experts), a structural compression pipeline that reduces Gemma 4 26B-A4B from 9.92 GB to a 4.75 GB GGUF container, enabling 100% in-VRAM execution on 6 GB laptop GPUs at 45.4 tokens/second. The MoE model originally contains 128 routed experts across 30 layers (25.6B total parameters); MAPLE profiles live forward-pass activations using llama-imatrix on a 1,979-token calibration corpus spanning Python, algorithms, data structures, and STEM reasoning. Experts are scored via a composite metric s_{e,l} = C_{e,l} · √(σ̄_{e,l} + ε), combining empirical routing frequency C_{e,l} and mean input channel activation energy σ̄_{e,l}. The pipeline excises 80 dormant experts per layer, retaining 48 experts that capture 79.88% of active forward-pass activation mass. The resulting 10.95B physical parameter container uses blended IQ2_XXS quantization at 3.47 physical bits/param (1.48 effective bits/param relative to the unpruned base). VRAM accounting shows 4,526 MiB static weights plus ~294 MiB KV cache and scratch buffers total 4,820 MiB, leaving 1,324 MiB headroom on a 6,144 MiB RTX 4050. Benchmarks report 6.57× speedup over the paged baseline (6.9 → 45.4 tok/s, 144.9 → 22.0 ms/token) with 93.8% mean keyword hit rate across Fibonacci synthesis, BST implementation, Rayleigh scattering derivation, and syllogistic logic. The pipeline avoids ghost-router proxy failures, cascaded normalization suppression (preserving scales at 1.0×), and DRY sampler interference. Code, pre-built GGUF (AmSach/Gemma-4-26B-A4B-MAPLE-Sub5GB-GGUF), and Zenodo citation (10.5281/zenodo.24042480) are provided under Apache 2.0.
Read original
hackernews