arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.09241cs.CL
收藏

何时信任:一种因果意识校准框架用于准确的知识图谱检索增强生成

When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation

RMIT University(皇家墨尔本理工大学)

Jing Ren, Bowen Li, Ziqi Xu, Xikun Zhang, Haytham Fayek, Xiaodong Li

AI总结:

Ca2KG通过整合反事实提示和重新评分机制,提升KG-RAG在高风险领域的可靠性与准确性。

摘要:

知识图谱检索增强生成(KG-RAG)通过整合来自知识图谱的结构化知识,扩展了RAG范式,使大语言模型(LLMs)能够执行更精确且可解释的推理。尽管KG-RAG在复杂任务中提高了事实准确性,但现有KG-RAG模型往往严重过度自信,即使检索到的子图不完整或不可靠,仍会产生高置信度的预测,这在高风险领域部署时引发了担忧。为了解决这个问题,我们提出了Ca2KG,一种面向KG-RAG的因果意识校准框架。Ca2KG集成了反事实提示,该提示揭示了知识质量和推理可靠性之间的检索依赖性不确定性,并结合基于面板的重新评分机制,使预测在干预下保持稳定。在两个复杂问答数据集上的广泛实验表明,Ca2KG在保持或甚至提升预测准确性的同时,能够一致地提高校准效果。

英文摘要:

Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs, enabling Large Language Models (LLMs) to perform more precise and explainable reasoning. While KG-RAG improves factual accuracy in complex tasks, existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable, which raises concerns for deployment in high-stakes domains. To address this issue, we propose Ca2KG, a Causality-aware Calibration framework for KG-RAG. Ca2KG integrates counterfactual prompting, which exposes retrieval-dependent uncertainties in knowledge quality and reasoning reliability, with a panel-based re-scoring mechanism that stabilises predictions across interventions. Extensive experiments on two complex QA datasets demonstrate that Ca2KG consistently improves calibration while maintaining or even enhancing predictive accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑