用于改进知识图谱增强大语言模型训练和评估的真实子图
Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs
AI总结：
SynthKGQA通过生成高质量的知识图谱问答数据集，提升知识图谱增强大语言模型的训练和评估效果。
摘要：
从图结构的知识库中检索信息代表了提高大语言模型事实性的一个有前途的方向。尽管已经提出了各种解决方案，但由于缺乏具有真实目标的具有挑战性的问答数据集，方法之间的比较变得困难。我们提出了SynthKGQA，一个由大型语言模型驱动的框架，可以从任何知识图谱生成高质量的知识图谱问答数据集，提供图谱中的完整事实集以推理问题。我们展示了如何，除了使知识图谱检索器的更信息丰富的基准测试成为可能外，通过SynthKGQA生成的数据还允许我们训练更好的模型。我们将SynthKGQA应用于Wikidata，生成GTSQA，一个新数据集，旨在测试知识图谱检索器对未见过的图结构和关系类型的零样本泛化能力，并在该数据集上对流行的知识图谱增强大语言模型解决方案进行基准测试。
英文摘要：
Retrieval of information from graph-structured knowledge bases represents a promising direction for improving the factuality of LLMs. While various solutions have been proposed, a comparison of methods is difficult due to the lack of challenging QA datasets with ground-truth targets for graph retrieval. We present SynthKGQA, an LLM-powered framework for generating high-quality Knowledge Graph Question Answering datasets from any Knowledge Graph, providing the full set of ground-truth facts in the KG to reason over questions. We show how, in addition to enabling more informative benchmarking of KG retrievers, the data produced with SynthKGQA also allows us to train better models.We apply SynthKGQA to Wikidata to generate GTSQA, a new dataset designed to test zero-shot generalization abilities of KG retrievers with respect to unseen graph structures and relation types, and benchmark popular solutions for KG-augmented LLMs on it.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
