RPO-RAG: 通过关系感知的偏好优化对齐小型LLM以实现知识图谱问答
RPO-RAG: Aligning Small LLMs with Relation-aware Preference Optimization for Knowledge Graph Question Answering
AI总结：
RPO-RAG通过关系感知的偏好优化和以答案为中心的提示设计，提升小型LLM在知识图谱问答中的推理能力，实现性能提升。
摘要：
大型语言模型（LLMs）最近在推理能力方面表现出色，但在知识密集型任务上容易产生幻觉。检索增强生成（RAG）通过将答案根植于外部来源，例如知识图谱（KGs）来缓解这一问题。然而，现有的基于KG的RAG方法依赖于语义不敏感的路径采样，并且与KG推理目标的对齐程度较低，这限制了进一步的准确性提升。它们还直接将检索到的路径输入推理器，而没有将它们组织成以答案为中心的推理路径，阻碍了小型LLM利用检索到的知识的能力。此外，先前的工作主要依赖于大型LLM（例如ChatGPT/GPT-4）或假设骨干在7B参数以上，使子7B模型未被充分探索。我们通过RPO-RAG填补这一空白，这是首个专门针对小型LLM设计的基于KG的RAG框架。RPO-RAG引入了三个关键创新：（1）一个查询路径语义采样策略，提供有信息的监督信号；（2）一种关系感知的偏好优化，使训练与中间KG推理信号（例如关系）对齐；（3）一种以答案为中心的提示设计，将实体和推理路径组织成可解释的格式。在两个基准知识图谱问答（KGQA）数据集，WebQSP和CWQ上进行了广泛的实验，证明RPO-RAG有效缩小了小型和大型语言模型之间的性能差距。在WebQSP上，它将F1提高多达8.8%，反映了更高的答案精度，而在CWQ上，它在8B参数以下的模型中实现了新的最先进结果。总体而言，RPO-RAG显著提高了小型LLM的推理能力，即使在3B参数以下的情况下，突显了它们在资源高效和实用的设备端KGQA应用中的潜力。
英文摘要：
Large Language Models (LLMs) have recently demonstrated remarkable reasoning abilities, yet hallucinate on knowledge-intensive tasks. Retrieval-augmented generation (RAG) mitigates this issue by grounding answers in external sources, e.g., knowledge graphs (KGs). However, existing KG-based RAG approaches rely on semantics-unaware path sampling and are weakly aligned with KG reasoning objectives, which limits further accuracy gains. They also feed retrieved paths directly into the reasoner without organizing them into answer-centered reasoning paths, hindering small LLMs' ability to leverage the retrieved knowledge. Furthermore, prior works predominantly rely on large LLMs (e.g., ChatGPT/GPT-4) or assume backbones above 7B parameters, leaving sub-7B models underexplored. We address this gap with RPO-RAG, the first KG-based RAG framework specifically designed for small LLMs, to the best of our knowledge. RPO-RAG introduces three key innovations: (1) a query-path semantic sampling strategy that provides informative supervisory signals; (2) a relation-aware preference optimization that aligns training with intermediate KG reasoning signals (e.g., relation); and (3) an answer-centered prompt design that organizes entities and reasoning paths in an interpretable format. Extensive experiments on two benchmark Knowledge Graph Question Answering (KGQA) datasets, WebQSP and CWQ, demonstrate that RPO-RAG effectively bridges the performance gap between small and large language models. On WebQSP, it improves F1 by up to 8.8%, reflecting enhanced answer precision, while on CWQ it achieves new state-of-the-art results among models under 8B parameters in both Hit and F1. Overall, RPO-RAG substantially improves the reasoning capability of small LLMs, even under 3B parameters-highlighting their potential for resource-efficient and practical on-device KGQA applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
