Moonshot AI has released Kimi-K3, an open-weight, native multimodal agentic model featuring 2.8 trillion total parameters and 104 billion activated parameters. Built on a Mixture-of-Experts (MoE) architecture using a Stable LatentMoE framework, the model activates 16 out of 896 experts per token, achieving a 2.5× scaling efficiency improvement over its predecessor, Kimi K2. The architecture incorporates Kimi Delta Attention (KDA), Attention Residuals (AttnRes), and Gated MLA, supported by a 401M parameter MoonViT-V2 vision encoder. Kimi-K3 supports a 1-million-token context window and utilizes quantization-aware training with MXFP4 weights and MXFP8 activations. In benchmarking, the model demonstrated frontier intelligence, scoring 93.5 on GPQA Diamond and 88.3 on Terminal-Bench 2.1. Its agentic capabilities are highlighted by a 91.2 score on BrowseComp and high performance in long-horizon coding tasks, such as achieving 81.2 on FrontierSWE. The model is designed for complex reasoning and knowledge work, providing native support for text, image, and video modalities, and is compatible with inference engines like vLLM and SGLang.
Read original
hackernews