Thinking Machines released Inkling 2 Mini, a 26.4B total parameter Mixture-of-Experts model with 3.4B active parameters and 5.4B weight activations per token due to its looped transformer architecture. The 24-layer model executes its two middle blocks (12 layers) three times per token while the first and last blocks run once, employing attention residuals and a learned gate to merge the previous loop's message with the prelude output. This design boosts serial reasoning depth without increasing parameters, achieving 48.2% on DeepSWE v1.1 and 46.5% on Vals Index v2.2, leading all open-weights models under 100B parameters. Trained natively in NVFP4 (20 layers) and MXFP8 (final 4 layers), the ~20GB footprint enables local execution on consumer laptops. A variable-loop research variant (TML-d24-varloop-research) demonstrated that no-CoT time horizon rises from 16 seconds (one pass) to 37 seconds (three passes), with serial reasoning (N-Hop Lookup) improving out to ten loops—double the training maximum. CoT length halves from 3,703 to 1,661 tokens between one and five passes while maintaining legibility (sentence length ~8.7 words, 94.4% dictionary words), showing more direct reasoning with less backtracking. Controllability remains near zero across loop counts. Knowledge-intensive benchmarks (AA-Omniscience, SimpleQA) are loop-invariant, while long-context retrieval (GraphWalks BFS 69.7%, MRCR v2 46.4%) and agentic search (BrowseComp 79.0%, DeepSearchQA 85.3%) improve. Stanford measured 0.703 intelligence-per-joule on DGX Spark, outperforming Qwen3.6-35B-A3B (0.430) and Gemma-4-26B-A4B (0.159). Training involved pruning two-thirds of 384 experts at midtraining, healing via 1T tokens of distillation, then SFT and 2.2T token RL. The non-looped baseline required 140% more inference compute to match looped loss on 512k agentic traces.
Read original
hackernews