Google releases specialized MTP drafters for Gemma 4, enabling massive speculative decoding speedups on consumer hardware.
What Multi-Token Prediction Drafters Actually Do
Standard language model inference is sequential: the model produces one token, feeds it back in, and produces the next. That loop is the main reason large models feel slow on local hardware, because each step pays the full cost of a forward pass regardless of how predictable the next word is. Speculative decoding breaks the loop by letting a small, fast "drafter" model guess several tokens ahead, which the main model then verifies in a single pass.
Gemma 4's Multi-Token Prediction (MTP) drafters are purpose-built for exactly this role. Instead of predicting one token at a time, an MTP drafter proposes a short run of upcoming tokens in one shot. The full model checks that draft in parallel and keeps every token that matches what it would have generated on its own, falling back to normal decoding only where the guess diverges.
Why This Yields a 3x Speedup Without Changing Output
The key property of speculative decoding is that it is lossless: the larger model still validates every token, so the text you get is identical to what the model would produce unassisted. The speedup comes entirely from doing more useful work per forward pass. When the drafter's guesses are accepted, several tokens land for the price of one verification step, and the 3x figure reflects how often those guesses hold up on typical prompts.
Because the drafters are specialized for Gemma 4 rather than being a generic small model bolted on, their predictions align more closely with the target model's distribution. Higher acceptance rates mean fewer wasted drafts and fewer fallbacks to slow single-token decoding, which is where most of the practical gain lives.
Running It on Consumer Hardware
The reason this matters for local setups is that consumer GPUs and even capable CPUs are usually memory-bandwidth bound during inference, not compute bound. You spend most of each step just moving the model's weights through memory. Verifying a batch of drafted tokens amortizes that cost across several tokens at once, so a machine that felt sluggish token-by-token can produce noticeably snappier responses.
- Pair the full Gemma 4 model with its matching MTP drafter rather than an unrelated small model, so acceptance rates stay high.
- Expect the largest wins on predictable text — code, structured output, and boilerplate — where the drafter guesses correctly more often.
- Budget for the drafter's own memory footprint; it is small, but it still shares your device's resources.
Where the Tradeoffs Show Up
Speculative decoding is not free in every situation. On highly unpredictable or adversarial prompts, the drafter's guesses get rejected more often, so you pay for drafts that never get used and the speedup shrinks toward baseline. The technique never makes output worse, but the acceleration is variable and depends heavily on how well the content matches what the drafter expects.
For most day-to-day local workloads, though, the math favors keeping the drafter on. It turns otherwise idle verification capacity into real throughput, and because it is tuned specifically for Gemma 4, it needs little tuning of your own to be worthwhile. If you run the model locally, enabling the MTP drafter is one of the lowest-effort ways to make it feel faster.