arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.26287cs.SE
收藏

赋能基于蒙特卡洛树搜索驱动的强化学习的RepoQA-Agent

Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search

Zhejiang University(浙江大学);Alibaba Group(阿里巴巴集团)

Guochang Li, Yuchen Liu, Zhen Qin, Yunkun Wang, Jianping Zhong, Chen Zhi, Binhua Li, Fei Huang, Yongbin Li, Shuiguang Deng

AI总结:

针对现有仓库级软件工程任务中LLM智能体方法的局限,提出由MCTS驱动的RepoSearch-R1强化学习框架,通过自训练生成推理轨迹,构建的RepoQA-Agent在答案完整性和训练效率上均有显著提升且无数据合规问题。

摘要:

仓库级软件工程任务要求大型语言模型(LLM)通过多轮工具交互,高效地从复杂代码库中导航并提取信息。现有方法存在显著局限性:免训练的上下文学习方法难以有效引导智能体根据环境反馈使用工具和做出决策,而基于训练的方法通常依赖从更大的LLM中进行成本高昂的蒸馏,会在企业环境中引发数据合规问题。为应对这些挑战,我们提出RepoSearch-R1,这是一种由蒙特卡洛树搜索(MCTS)驱动的新型智能体强化学习框架。该方法无需模型蒸馏或外部监督,就能让智能体通过自训练生成多样化、高质量的推理轨迹。基于RepoSearch-R1,我们构建了专门面向仓库问答任务的RepoQA-Agent。在仓库问答任务上的综合评估表明,RepoSearch-R1在答案完整性上实现了大幅提升:相比无检索方法提升16.0%,相比迭代检索方法提升19.5%,且与通用智能体强化学习方法相比训练效率提升33%。我们的冷启动训练方法消除了数据合规顾虑,同时在仓库级推理任务中保持了稳健的探索多样性和答案完整性。

英文摘要:

Repository-level software engineering tasks require large language models (LLMs) to efficiently navigate and extract information from complex codebases through multi-turn tool interactions. Existing approaches face significant limitations: training-free, in-context learning methods struggle to guide agents effectively in tool utilization and decision-making based on environmental feedback, while training-based approaches typically rely on costly distillation from larger LLMs, introducing data compliance concerns in enterprise environments. To address these challenges, we introduce RepoSearch-R1, a novel agentic reinforcement learning framework driven by Monte-carlo Tree Search (MCTS). This approach allows agents to generate diverse, high-quality reasoning trajectories via self-training without requiring model distillation or external supervision. Based on RepoSearch-R1, we construct a RepoQA-Agent specifically designed for repository question-answering tasks. Comprehensive evaluation on repository question-answering tasks demonstrates that RepoSearch-R1 achieves substantial improvements of answer completeness: 16.0% enhancement over no-retrieval methods, 19.5% improvement over iterative retrieval methods, and 33% increase in training efficiency compared to general agentic reinforcement learning approaches. Our cold-start training methodology eliminates data compliance concerns while maintaining robust exploration diversity and answer completeness across repository-level reasoning tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑