The researchers introduce $\beta$-OPSD, a novel framework designed to improve the reliability and performance of on-policy self-distillation (OPSD) for reasoning language models. The authors identify that standard OPSD is a specific case of a broader policy-optimization family where the KL penalty weight, $\beta$, is fixed at 1. By treating $\beta$ as a controllable regularization parameter, the authors derive an optimal policy that serves as a geometric interpolation between the reference policy and a privileged teacher. To avoid the high variance and computational costs associated with direct reinforcement learning optimization of this objective, the method utilizes a distillation-based approach. Specifically, $\beta$-OPSD approximates the closed-form solution by using the reference and teacher token-level logits as distillation targets, effectively mixing them to create a target along the reference-to-teacher path. This process is augmented with return-to-go credit assignment to align token-level updates with sequence-level objectives. Experimental evaluations on mathematical reasoning benchmarks demonstrate that $\beta$-OPSD provides superior optimization stability and higher downstream reasoning performance compared to vanilla OPSD, offering a principled method to bridge the gap between efficient self-distillation and robust policy optimization.