Research indicates that RL training for LLMs is often unstable due to a training-inference mismatch, where separate engines produce inconsistent probabilities for the same trajectories. The authors propose that monotonic inference policies should be the primary objective rather than optimizing training policies to address this instability and prevent model collapse.