The researchers introduce ABSeeker, a framework designed to train long-horizon search agents by addressing the limitations of uniform credit assignment in supervised fine-tuning (SFT) and reinforcement learning (RL). To solve the issue where sparse, trajectory-level outcomes fail to distinguish productive actions from redundant ones, the authors propose Answer-Backtracked Credit Assignment (ABC). This method utilizes Answer-Backtracked Clue Recovery to trace intermediate clues from a ground-truth answer back through the search process. These clues enable Clue-Anchored Step Scoring, which transforms binary success metrics into dense, step-level rewards. The researchers implemented two training variations: ABC-SFT, which reweights losses per turn, and ABC-GRPO, which incorporates step-level scores into the Group Relative Policy Optimization algorithm. Training a Qwen3.5-4B model on a small dataset of 8.5k examples yielded significant results. ABSeeker achieved 37.3% on BrowseComp and 39.1% on BrowseComp-ZH, which increased to 55.3% and 52.9% respectively when using context management. Notably, this 4B-parameter agent's performance matched that of much larger models, approximately 30B parameters, demonstrating that dense, clue-anchored supervision is highly effective for complex, multi-step retrieval and verification tasks.
Read original
huggingface/daily-papers