Thinking Machines Lab released Inkling 2 Mini, a 26.4B-parameter sparse Mixture-of-Experts model employing a looped transformer architecture that reuses its two middle 6-layer blocks three times per token, yielding 48 effective layer passes from 24 stored layers. This design achieves 3.4B active parameters (5.4B effective per token with looping) within a ~20GB checkpoint, enabling local deployment on Apple Silicon with 32GB+ memory, a single NVIDIA DGX Spark, or GPUs with 32GB+ VRAM. The model supports up to 1M-token context, accepts text and image inputs, and is licensed Apache 2.0 on Hugging Face with hosted inference via Tinker. Training began with 384 experts per layer (8 active), pruned mid-training to 128 experts, followed by a 1T-token distillation from a larger internal model, then SFT and RL over 2.2T tokens. Experts deploy in NVFP4; the final four layers use MXFP8 to eliminate quantization conversion loss. Self-reported benchmarks show strength in agentic coding: 79.2% on SWE-bench Verified (bash only), 77.9% on SWE-bench Pro v2 Hard, and 48.2% on DeepSWE v1.1 (vs. Qwen3.8-27B's 42.2%). It scores 71.6% on Terminal Bench v2.1, 82.7% on MCP Atlas, 92.8% on OmniDocBench v1.6, and 86.6% on GPQA Diamond. Efficiency claims include 0.703 Intelligence per Watt on DGX Spark, exceeding Qwen3.6-35B-A3B (0.430) and Gemma-4-26B-A4B (0.159). Weaknesses include knowledge recall (15.2% HLE-Diamond, -8.6 AA-Omniscience, 63.4% non-hallucination rate) and multi-turn instruction following (50.0% MultiChallenge, 33.4% ComplexConstraints v1.1). Quantized GGUF builds are available via Unsloth and llama.cpp; serving supports Transformers, vLLM, SGLang, Ollama, NVIDIA Dynamo, and TokenSpeed.

Read original