FlashPrefill V2 advances long-context Large Language Model (LLM) serving by transitioning a previous algorithmic prototype into a production-ready sparse attention framework. To address the quadratic complexity of the prefill phase, the researchers introduced a mean correction term that minimizes approximation errors during high-sparsity operations. The architecture features a redesigned sparse attention operator utilizing PackGQA memory access, warp specialization, and pingpong pipelining to maintain alignment with FlashAttention-3/4 standards. Notably, the system supports FP8 inference and integrates seamlessly with modern serving frameworks via native support for paged KV cache and continuous batching. Experimental evaluations conducted on NVIDIA H20 GPUs at a 128K context length reveal significant performance gains: FlashPrefill V2 achieves up to 47.26x speedup over FlashAttention-2 when using FP8 precision, and a 27.19x speedup using BF16. Furthermore, in FP8 mode, the method maintains a 30.49x speedup compared to a dense baseline aligned with FA3/4. By combining efficient pattern discovery with hardware-aware optimizations, FlashPrefill V2 provides a scalable solution for high-throughput, long-context inference deployment.

Read original