This research investigates the limitations of applying Classifier-Free Guidance (CFG) within On-Policy Distillation (OPD) for diffusion models. Standard OPD approaches typically employ velocity matching on the CFG-composed prediction, assuming that minimizing errors in the final guided output is sufficient. However, the authors demonstrate that this objective is mathematically under-identified at the branch level. They identify a failure mode termed Negative Branch Asymmetry (NBA), which occurs when the teacher model's negative branch contains privileged information unavailable to the student. In these instances, the composed objective creates antagonistic dynamics where reducing the positive-branch error inadvertently increases the negative-branch error. To mitigate this, the researchers propose Positive-Direction Matching (PDM), a branch-aware distillation objective. Unlike naive matching, PDM separately constrains the positive prediction and the CFG conditional direction to ensure stable learning. When applied to dense-to-sparse video control tasks, the PDM method proves significantly more robust and effective than standard guided matching, which remains highly sensitive to variations in inference guidance scales. By decoupling the branch errors, PDM facilitates more reliable knowledge transfer from the teacher to the student during the distillation process.

Read original