DeepSeek-V4.1 Flash is a 552B parameter multimodal mixture-of-experts model engineered to maximize KV Cache compression for long-horizon agent workflows. To address the storage and communication bottlenecks of long contexts, the architecture implements a Causal Encoder-Decoder (CED) framework inspired by YOCO. This design splits the 40-layer model into a 20-layer Causal Encoder and a 20-layer Decoder; during the prefill stage, only the first 20 layers are activated, reducing activated parameters to 8B compared to 16B during decoding. The model utilizes CSA2 (Compressed Sparse Attention) to compress the KV Cache across three dimensions: the channel dimension via a 512-dimensional latent vector, the sequence dimension by merging adjacent positions in the Encoder, and the layer dimension by sharing global KV across multiple layers. Combined with FP4 quantization, these optimizations achieve a 4x reduction in runtime KV Cache storage and an 8x reduction in persistent storage compared to DeepSeek-V4-Flash. Additionally, the model incorporates a "Decoder SWA Bounded Replay" to limit the computational overhead of local Sliding Window Attention and uses DSpark speculative decoding to maintain high throughput, reaching speeds of nearly 420 Tokens/s.
Read original
hackernews