可靠的代码库图-RAG:基于AST的图与LLM提取的知识图谱
Reliable Graph-RAG for Codebases: AST-Derived Graphs vs LLM-Extracted Knowledge Graphs
AI总结:
本文比较了基于AST的图谱和LLM提取的知识图谱在代码库中的检索性能,发现确定性AST图谱在索引成本和多跳推理方面更具优势。
摘要:
在软件工程中,检索增强生成(RAG)通常依赖于向量相似性搜索,这种技术能够捕捉到主题相似性,但在涉及多跳架构推理的问题上(如控制器到服务到仓库的链式结构、接口驱动的布线、继承关系)可能会失败。本文在Java代码库(Shopizer,此外还包括在ThingsBoard和OpenMRS Core上的额外运行)上评估了三种检索流程:(A)仅向量的无图RAG(No-Graph RAG)、(B)由LLM生成的知识图谱RAG(LLM-KB)以及(C)由Tree-sitter和双向遍历构建的确定性AST衍生知识图谱RAG(DKB)。使用每个仓库15个架构和代码追踪查询,我们测量了索引时间、查询延迟、语料覆盖率、成本和回答正确性。DKB在几秒钟内构建其图谱,而LLM-KB需要更长的图谱生成时间。LLM-KB还显示出索引不完整:在Shopizer中,377个文件被跳过或遗漏,导致嵌入块覆盖率和图谱大小比DKB小。DKB的端到端成本相对于仅向量的基线来说相对较低,但LLM-KB的成本则更高,尤其是在代码库规模增加时。查询延迟在无图和DKB之间相似,而LLM-KB则更慢且更不稳定。在Shopizer问题集上,DKB实现了最高的正确性,LLM-KB紧随其后,而仅向量的基线在上游架构查询上表现最差且有最高的幻觉风险。总体而言,确定性AST衍生图谱在显著较低的索引成本下提供了比LLM提取图谱更可靠和多跳支撑。
英文摘要:
Retrieval-Augmented Generation for software engineering often relies on vector similarity search, which captures topical similarity but can fail on multi-hop architectural reasoning such as controller to service to repository chains, interface-driven wiring, and inheritance. This paper benchmarks three retrieval pipelines on Java codebases (Shopizer, with additional runs on ThingsBoard and OpenMRS Core): (A) vector-only No-Graph RAG, (B) an LLM-generated knowledge graph RAG (LLM-KB), and (C) a deterministic AST-derived knowledge graph RAG (DKB) built with Tree-sitter and bidirectional traversal. Using 15 architecture and code-tracing queries per repository, we measure indexing time, query latency, corpus coverage, cost, and answer correctness. DKB builds its graph in seconds, while LLM-KB requires much longer graph generation. LLM-KB also shows indexing incompleteness: on Shopizer, 377 files are skipped or missed, reducing embedded chunk coverage and graph size compared to DKB. End-to-end cost is modest for DKB relative to the vector-only baseline but much higher for LLM-KB, especially as repository scale increases. Query latency is similar for No-Graph and DKB, while LLM-KB is slower and more variable. On the Shopizer question suite, DKB achieves the highest correctness, LLM-KB is close behind, and the vector-only baseline performs worst on upstream architectural queries and has the highest hallucination risk. Overall, deterministic AST-derived graphs provide more reliable coverage and multi-hop grounding than LLM-extracted graphs at substantially lower indexing cost.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。