This research investigates the efficacy of On-Policy Distillation (OPD), questioning whether student improvement stems from teacher supervision or inherent self-improvement mechanisms. The authors demonstrate that teacher scoring of student-generated trajectories introduces significant noise, which actually increases alongside teacher scale. Quantitative analysis reveals that the student policy remains largely insensitive to this noise, suggesting that OPD's performance gains are not driven by teacher distillation. Instead, the process primarily functions by suppressing low log-probability tokens. The study finds that using a single fixed negative advantage achieves performance comparable to complex teacher-provided advantages, confirming that the primary mechanism is the reduction of tail token probabilities. To leverage this insight, the authors propose On-Policy Self-Adaptation (OPSA), a supervision-free method that utilizes entropy-adaptive negative advantages. OPSA assigns stronger learning signals to high-entropy positions to suppress tail tokens while redistributing probability mass to head tokens. Experimental results show that OPSA significantly outperforms the base Qwen3-1.7B, achieving a 35.41 point increase in Avg@32 on the AIME24 benchmark—a 263% relative gain—and more than doubling the Pass@32 metric across three benchmarks. Furthermore, OPSA outperforms the standard OPD approach by 16.77 points in Avg@32 on AIME24, demonstrating superior effectiveness and generalizability through self-adaptation.