The researchers introduce SimpleOPD, a tokenizer-agnostic on-policy distillation (OPD) framework designed to transfer long-context reasoning capabilities from powerful teachers, such as the SU-01 model, to short-context student models. To overcome the challenges of tokenizer mismatch, the method performs distillation within a shared text space, aligning only those tokens that correspond to identical text spans across both models. To address distribution mismatch and the tendency for student models to produce excessively long responses that trigger truncation, the authors implement a student reference KL loss and mask the advantages of special termination tokens like </think> and <|im_end|>. This approach stabilizes training and encourages gradual length growth. Empirical evaluations on diverse architectures, including Qwen3, Intern-S2, GLM-4.7, and Gemma-4, demonstrate significant improvements in mathematical reasoning and natural-language math proving. Most notably, the Intern-S2-Preview model achieved a 21.2-point increase on the ProofBench benchmark, reaching a score of 55.2 and outperforming Gemini-2.5-Pro. Furthermore, the method shows generalized benefits across scientific benchmarks such as HLE and HiPhO, indicating that the distilled reasoning capabilities extend beyond the specific mathematical domain used during training.
Read original
huggingface/daily-papers