The paper introduces Self-Credit , a novel agentic reinforcement learning (RL) framework designed to address sparse reward challenges in long-horizon tasks. Unlike traditional RLVR methods, Self-Credit generates dense supervision signals by leveraging self-generated credit assignment through intra-group rescaling. The approach comprises three stages: (1) a policy model produces trajectory summaries with citations to prior turns containing critical evidence, (2) intra-group rescaling refines these signals by contrasting cited actions between successful and failed rollouts for identical prompts, using group-level agreement/disagreement to suppress undeserved credit and amplify relevant actions, and (3) advantage reweighting adjusts cited turn advantages using rescaled weights. Evaluated on BrowseComp-Plus, Self-Credit boosts performance over base models by 37.0 and 36.2 points for Qwen3 8B and 32B, respectively, and surpasses strong agentic RL baselines by 8.5 and 5.6 points. It also achieves 4.9-point and 4.7-point gains over baselines on GAIA and xBench-DeepSearch. Analyses confirm that self-generated credit aids critical action selection, while intra-group rescaling, particularly on failed rollouts, proves vital across model scales. The method demonstrates robustness in sparse-reward settings by iteratively refining credit signals through group-based self-evaluation.
Read original
huggingface/daily-papers