发表机构
Shanghai Jiao Tong University（上海交通大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出答案回溯信用分配框架，训练ABSeeker智能体，在BrowseComp等数据集上性能优于同规模模型，接近30B级大模型。
AI 中文摘要
长程搜索智能体必须执行多个连续动作（步骤）来搜索、检索、验证和整合证据，以得出最终答案。然而，现有的训练这些智能体的方法在监督微调（SFT）和强化学习（RL）过程中，通常对轨迹内的所有步骤一视同仁，无法区分有用动作与错误或冗余动作。本文提出了答案回溯信用分配（ABC），这是一种用于训练长程搜索智能体的细粒度信用分配框架，通过将稀疏的轨迹级结果转换为密集的步骤级监督，奖励有用动作（即使在失败的轨迹中），同时抑制错误或冗余动作。具体而言，给定一个可能模糊的查询及其对应的真实答案，ABC首先执行答案回溯线索恢复，即从答案回溯以恢复解决问题所需的中间线索；随后应用线索锚定步骤评分，根据这些线索评估每个搜索步骤，将稀疏的二元结果监督转换为密集的步骤级奖励。基于这些奖励，我们开发了ABC-SFT（对每一轮的损失进行加权）和ABC-GRPO（将步骤级评分作为GRPO中的奖励）。基于该框架，我们仅用8500个示例在Qwen3.5-4B的基础上训练了ABSeeker。ABSeeker在BrowseComp上的准确率达到37.3%，在BrowseComp-ZH上达到39.1%；加入上下文管理后，分数进一步提升至55.3%和52.9%，显著优于同规模（4B）智能体，甚至可与更大规模（约30B）智能体的性能相媲美。这些结果证明了答案回溯步骤级信用分配在训练长程搜索智能体方面的有效性。
英文摘要
Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.
