Moonshot AI has released Kimi K3.1 Pro, a 2.8 trillion parameter Mixture-of-Experts (MoE) model with 104 billion activated parameters, marking the first model in the K3.1 series. The architecture comprises 93 layers, including 69 KDA and 24 Gated DeepSeek Sparse Attention (Gated DSA) layers, replacing Kimi K3's Gated MLA for efficient long-context modeling. It features 896 experts with 16 selected per token, a 3,584 latent MoE dimension, SiTU-GLU activation, and a 160K vocabulary. The model supports a 1,048,576-token context window and natively handles text and image modalities via the 401M parameter MoonViT-V2 vision encoder. Weights are quantized to MXFP4 with MXFP8 activations through quantization-aware training. Post-trained from Kimi K3 via large-scale joint reinforcement learning across hundreds of domains, Kimi K3.1 Pro shows strong agentic performance: 65.7% on Terminal-Bench 4.0, 51.7% on SWE Marathon v1.1, 92.7% on BenchCAD with tools, and 75.5% on DeepSWE v1.1. On agentic benchmarks, it scores 70.0% on HLE-Full with tools, 92.1% on BrowseComp, 79.6% on OSWorld V2.1, and achieves a 1846 Elo on GDPval-AA v2.1. Reasoning scores include 94.2% on GPQA-Diamond and 51.1% on HLE-Full. The model is accessible via the Kimi platform API (OpenAI/Anthropic-compatible) and runs on vLLM, SGLang, and Docker Model Runner, with reasoning effort configurable via "low," "high," or "max" settings.

Read original