SpecOPD introduces branch-parallel on-policy distillation to overcome acceptance-length plateaus in speculative decoding. By parallelizing draft-and-verify rollouts across multiple branches, it enables efficient collection of on-policy supervision for drafter training. This approach pushes well-trained drafters beyond the limitations of conventional sequential OPD pipelines, improving overall inference speedup without increasing proposal cost.

Read original