A new paper titled "No Second Run: Does RSI Stop at Frontier Pre-training?" investigates whether recursive self-improvement (RSI) can be applied to the pre-training phase of foundation models. Current RSI research focuses on post-training, which allows for repeatable, verifiable experiments at lower computational cost. Pre-training presents fundamental barriers: runs consume hundreds of thousands of GPU-hours over months, cannot be repeated, and lack immediate verifiable scores. The paper's release coincides with the upcoming 801-30B-A3B-Base model.

Read original