arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.12276cs.AIcs.CLcs.SE
收藏

CodeNav:超越工具使用,借助LLM智能体使用真实世界代码库

CodeNav: Beyond tool-use to using real-world codebases with LLM agents

Allen Institute for AI(艾伦人工智能研究所)

Tanmay Gupta, Luca Weihs, Aniruddha Kembhavi

AI总结:

本研究提出CodeNav智能体,可自动索引搜索陌生代码库、调用代码片段并结合执行反馈迭代解题,通过案例与基准测试验证其效果优于依赖手动工具注册的传统工具使用型LLM智能体,相关代码将开源。

摘要:

我们提出了CodeNav,这是一种LLM(大语言模型)智能体,能够导航并利用此前未见过的代码仓库来解决用户查询。与需要在LLM上下文中通过手动描述完成所有相关工具“注册”的工具使用型LLM智能体不同,CodeNav会自动对目标代码库中的代码块进行索引和搜索,找到相关代码片段,导入它们,并借助执行反馈迭代生成解决方案。为凸显CodeNav的核心能力,我们首先展示了三个案例研究,其中我们使用CodeNav借助三个不同的代码库解决复杂的用户查询。接下来,在三个基准测试上,我们定量比较了代码使用(仅能访问目标代码库)与工具使用(拥有所有工具名称和描述的特权访问权限)的有效性。最后,我们研究了不同类型的工具和库描述对代码使用性能的影响,同时探究了智能体查看源代码相较于查看代码的自然语言描述所具备的优势。所有代码将以宽松许可协议开源。

英文摘要:

We present CodeNav, an LLM agent that navigates and leverages previously unseen code repositories to solve user queries. In contrast to tool-use LLM agents that require ``registration'' of all relevant tools via manual descriptions within the LLM context, CodeNav automatically indexes and searches over code blocks in the target codebase, finds relevant code snippets, imports them, and uses them to iteratively generate a solution with execution feedback. To highlight the core-capabilities of CodeNav, we first showcase three case studies where we use CodeNav for solving complex user queries using three diverse codebases. Next, on three benchmarks, we quantitatively compare the effectiveness of code-use (which only has access to the target codebase) to tool-use (which has privileged access to all tool names and descriptions). Finally, we study the effect of varying kinds of tool and library descriptions on code-use performance, as well as investigate the advantage of the agent seeing source code as opposed to natural descriptions of code. All code will be made open source under a permissive license.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑