A developer demonstrated running Qwen3.6-35B-A3B (Q4_K_M quantized) on a smartphone using stock llama.cpp in airplane mode, achieving 1.3 to 6.8 tokens per second over a ten-minute inference period. The model employs a Mixture of Experts architecture with 35B total parameters but only 3B active per token, storing frequently used components in RAM and loading expert weights from flash storage as needed. The author created an open-source "Pocket Interpreter" project to showcase this mobile inference capability.

Read original