An engineering deep-dive explores speculative decoding, a popular LLM inference optimization technique that pairs a small draft model with a larger target model to achieve 2–3× decoding throughput. The author implements the technique from scratch without using assisted generation libraries and benchmarks it on real model weights. The results challenge conventional assumptions, revealing that speculative decoding was slower than expected in practice, highlighting the gap between theoretical promises and real-world performance.

Read original