Speculative Decoding: Faster LLM Inference Without Quality Loss
Speculative decoding accelerates large language model inference by drafting several tokens cheaply and validating them with a single target-model pass. A rejection-sampling step ensures the resulting distribution matches…
→ View original source