A developer pruned the Gemma 4 26B-A4B MoE model to 15% of its original size, retaining 92% output agreement with the full model on a specific classification task. This performance matches the 90% agreement rate observed between the 12B and 26B parameter variants. The pruned model is unsuitable for general tasks or knowledge-intensive workloads due to information loss but serves a specific on-device application requiring a minimal 24/7 memory footprint.

Read original