PersonalAI 2.0:通过规划机制增强知识图谱遍历/检索以实现个性化大语言模型代理
PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents
AI总结:
PersonalAI 2.0通过动态多阶段查询流程提升知识图谱遍历与检索能力,结合实体提取和生成线索查询,提升回答准确性,优于现有方法,且在多个基准测试中表现优异。
摘要:
我们引入PersonalAI 2.0(PAI-2),一种新型框架,旨在通过整合外部知识图谱(KG)增强基于大语言模型(LLM)的系统。所提出的方法通过引入动态、多阶段查询处理流程,解决了现有图检索增强生成(GraphRAG)方法的关键限制。PAI-2的核心设计是其能够通过提取的实体、匹配的图顶点和生成的线索查询进行自适应、迭代信息搜索。在六个基准测试(Natural Questions、TriviaQA、HotpotQA、2WikiMultihopQA、MuSiQue和DiaASQ)上的评估表明,与类比方法(LightRAG、RAPTOR和HippoRAG 2)相比,PAI-2在事实正确性方面有所提高。PAI-2通过LLM-as-a-Judge在四个基准测试中平均获得4%的提升,反映了其在减少幻觉率和提高精度方面的有效性。我们显示,使用图遍历算法(例如BeamSearch、WaterCircles)相比标准扁平检索器平均获得6%的更好结果,而启用搜索计划增强机制相比禁用的机制在六个数据集上通过LLM-as-a-Judge获得18%的提升。此外,消融研究显示,PAI-2在MINE-1基准测试中达到SOTA结果,使用7-14B级LLM,获得89%的信息保留分数。这些发现强调了PAI-2作为下一代个性化AI应用基础模型的潜力,需要可扩展、上下文感知的知识表示和推理能力。
英文摘要:
We introduce PersonalAI 2.0 (PAI-2), a novel framework designed to enhance LLM-based systems through integration of external knowledge graphs (KGs). The proposed approach addresses key limitations of existing Graph Retrieval-Augmented Generation (GraphRAG) methods by incorporating a dynamic, multistage pipeline. The central point of the PAI-2 design is its ability to perform adaptive, iterative information search, guided by extracted entities, matched graph vertices and generated clue-queries. Conducted evaluation on five benchmarks (Natural Questions, TriviaQA, HotpotQA, 2WikiMultihopQA and MuSiQue) demonstrates an improvement in the factual correctness of generated answers compared to baseline methods (LightRAG, RAPTOR, HippoRAG 2 and PAI-1). PAI-2 achieves a 9 percentage points (pp) average gain by LLM-as-a-Judge on 2WikiMultihopQA and MuSiQue benchmarks, and attains comparable accuracy to HippoRAG 2 on the TriviaQA and HotpotQA benchmarks with Qwen2.5 7B and multilingual-e5-large reflecting its effectiveness in reducing hallucination rates and increasing recall. We show that enabled search plan enhancement mechanism gain in average a 18 pp boost compared to disabled one by LLM-as-a-Judge across five benchmarks. In addition, an ablation study reveals that PAI-2 achieves superior result on the MINE-1 benchmark, obtaining an 89\% information-retention score with Qwen2.5 7B and multilingual-e5-large. Collectively, these findings underscore the potential of PAI-2 to serve as a reusable component for personalized AI applications, which require context-aware knowledge-representation and reasoning capabilities. The source code of PAI-2 is available at the following link: https://github.com/Dzigen/PersonalAI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。