The paper introduces speculative decoding as a method to accelerate autoregressive LLM inference by using a faster draft model, traditionally requiring lossless acceptance relative to a verifier distribution, whereas recent diffusion LLMs achieve speedups by deliberately lowering output quality through reduced diffusion steps. The authors formalize a joint optimization of two controllable factors — diffusion effort, measured by the number of diffusion sampling steps, and speculative decoding leniency, defined by the acceptance threshold relative to the verifier distribution — and construct the full achievable Pareto frontier of speed versus quality. Empirical evaluation on multiple LLM benchmarks shows that jointly varying these parameters yields speed gains beyond those attainable with single‑step diffusion drafting, while allowing controlled degradation of quality. By sweeping diffusion effort from 1 to 10 steps and leniency from strict to relaxed acceptance, the study demonstrates that the frontier can be shifted to provide higher throughput at comparable quality or higher quality at modest additional compute. The work thus clarifies that the tradeoff between diffusion effort and speculative acceptance is the primary lever for optimizing speculative diffusion, offering a systematic way to balance latency and fidelity in LLM inference.

Read original