A study linking commit histories from seven open-source reinforcement-learning post-training frameworks to 6,939 screened arXiv papers from January 2025 through June 2026 assessed how readily published RL methods become usable in practice. Of 1,598 papers proposing general RL training methods, only 37 were incorporated into at least one framework. Across five quarterly paper cohorts, the number implemented within 180 days remained at 5–7 even as cohort size expanded from 72 to 386 papers, indicating no corresponding rise in framework uptake. Included work was more likely to train policies with at least 30 billion parameters—41% versus 10% for papers from the same quarters—and to adjust a coefficient, clipping value, mask, or weight in an existing loss—29% versus 9%. These differences persisted after excluding papers authored by organizations maintaining the frameworks, while code-release rates were similar between groups. Framework issue and pull-request records suggest that exclusion is rarely explicit: more than 90% of general methods never appeared by name in public discussions, and many threads received no response. Publicly reported training failures were also uncommon. Technical reports for 13 large open-weight models documented eight widely included methods. The authors released a tracker connecting failure reports with framework implementations, contribution policies, and earlier proposals.
Read original
huggingface/daily-papers