To address the inference bottlenecks inherent in Diffusion Large Language Models (dLLMs), the researchers introduced Flash-dLLM, a training-free acceleration framework designed to mitigate GPU memory I/O constraints. Recognizing that existing methods often fail to account for the I/O bottlenecks caused by simultaneous cache reuse and parallel token verification, the authors developed an I/O-aware fused KV-cache kernel to minimize redundant memory movement. This optimized mechanism supports a novel KV-cache-driven draft-and-verify decoding strategy. Unlike traditional methods that require an auxiliary model, Flash-dLLM utilizes the dLLM itself to perform both drafting and verification roles. This unified approach enhances scalability for larger batch sizes and longer sequences while maintaining generation quality. Empirical evaluations on mathematical reasoning and code-generation tasks demonstrate significant performance gains over current state-of-the-art acceleration techniques. Specifically, Flash-dLLM achieved a 5.1× speedup compared to the Elastic-Cache baseline on the GSM8K benchmark and an 11.0× speedup on the HumanEval benchmark, proving its effectiveness in improving both inference speed and memory efficiency for non-autoregressive text generation.
Read original
huggingface/daily-papers