MTP in Practice: Benchmarking Gemma's Speculative Decoding on a Real GPU

Article automatically generated from technical news.

In the concept edition, we saw that MTP (Multi-Token Prediction) lets a model predict several tokens ahead to speed up generation, and that Qwen and Gemma implement this in completely different ways. The theory makes sense, but how much faster does this actually make things in practice? That's what we set out to measure directly. In this installment — the implementation/benchmark edition — we'll actually run Gemma's MTP on llama.cpp and measure, with real numbers on an ordinar