The article describes running Qwen3.8-Flash-Next—a 125B-parameter MoE model activating ~6B parameters per token with a 51B-parameter n-gram embedding table—on a 128 GB Mac Studio M4 Max. The expert-pruned REAP-288 build (73.5 GB) lost nearly all Japanese and general knowledge despite its HumanEval score dropping only from 93.9 to 91.5, scoring 12/32 on Japanese tasks versus the full build's 32/32. The full 4-bit build (111.5 GB) exceeded available memory. The solution was memory-mapping the 32 GB n-gram table on demand via mlx-vlm's existing PLE infrastructure, reducing peak memory to 79.5 GB after loading and enabling 240K-token prompts peaking at 96 GB. Quality was fully preserved: Japanese 32/32, agentic 4/4, and long-context retrieval 3/3 at 32K/128K/240K. Prefill throughput reached 530–580 tok/s regardless of length, processing a 240K prompt in 7.5 minutes versus 35.2 minutes for the dense 27B (~5× faster). The fix was integrated into the author's home inference server kiapi, and upstream patches were submitted to mlx-vlm (PR #2309) to resolve image-cache reuse, improving token reuse from 0/1,333 to 1,047/1,333.

Read original