Researchers propose a unified training framework that consolidates the standard foundation model lifecycle — pretraining, midtraining, supervised fine-tuning, and reinforcement learning — into a single maximum-likelihood algorithm by reformulating RL objectives as fitting model-generated target distributions. This approach eliminates algorithmic stage boundaries, reframing training as a data scheduling problem over static and RL-generated data within one trainer. In controlled and real-world experiments, unified training improves the accuracy-compute tradeoff and task coverage while preserving broad capabilities, and yields a stronger initialization for continual learning. The system mixes static datasets and on-policy generations during optimization, demonstrating that staged training emerges as a special case of this generalized likelihood-based paradigm. The work establishes a mathematical and infrastructural foundation for a unified science of foundation model development, linking optimization principles, training systems, and capability progression.

Read original