Qwen has released Qwen3.8-Flash-Next, an experimental open-weight multimodal architecture preview designed to underpin the upcoming Qwen4. This causal language model utilizes a Mixture-of-Experts (MoE) design with 125B total parameters, featuring 6B activated parameters, 51B n-gram embedding parameters, and 4B multi-token-prediction (MTP) parameters. Architecturally, the model employs 48 layers consisting of 12 repetitions of three Gated DeltaNet-to-MoE blocks followed by one Qwen Sparse Attention (QSA)-to-MoE block. The QSA mechanism optimizes long-context latency by selecting micro-blocks rather than individual tokens. The model supports a native context window of 262,144 tokens, extensible to 1,000,000 tokens, and includes an integrated vision encoder for multimodal tasks. Benchmarks highlight its strength in agentic and coding workflows, scoring 91.9 on LiveCodeBench v6, 81.0 on SWE-bench Multilingual, and 73.9 on CoWorkBench. It also demonstrates advanced reasoning and visual capabilities, achieving 91.7 on GPQA Diamond and 88.5 on RealWorldQA. While designed for long-horizon tool use and software engineering, the model operates in a default "thinking mode," emitting reasoning within <think> tags. Deployment requires significant memory due to the 125B total parameters, despite the low activated count, and the current license is listed as "other," requiring legal review for commercial use.
Read original
dev.to