arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.05893cs.AIcs.CLcs.IR
收藏

使用结构化知识库增强大语言模型的元数据治理能力

Use of a Structured Knowledge Base Enhances Metadata Curation by Large Language Models

Stanford University(斯坦福大学)

Sowmya S. Sundaram, Benjamin Solomon, Avani Khatri, Anisha Laumas, Purvesh Khatri, Mark A. Musen

AI总结:

本研究针对大语言模型无辅助时元数据标准遵循度提升有限的问题,通过结合CEDAR模板等结构化知识库领域信息,使GPT-4的元数据标准遵循准确率从79%显著提升至97%,验证了结构化知识库对LLM自动化元数据治理的增强作用。

摘要:

元数据在确保数据集的可发现性、可访问性、互操作性和可重用性方面发挥着关键作用。本文研究了大语言模型(LLMs),具体为GPT-4,在提升元数据标准遵循度方面的潜力。我们对来自NCBI BioSample存储库的200条描述肺癌相关人类样本的随机数据记录开展了实验,评估GPT-4为遵循元数据标准提出编辑建议的能力。我们通过同行评审流程计算了字段名-字段值对的遵循准确率,观察到标准数据字典遵循度从79%小幅平均提升至80%(p<0.5)。随后我们以CEDAR模板文本描述的形式向GPT-4提供领域信息,记录到遵循度从79%显著提升至97%(p<0.01)。这些结果表明,尽管LLMs在无辅助时可能无法修正遗留元数据以确保令人满意的标准遵循度,但当与结构化知识库集成时,它们在自动化元数据治理中展现出应用前景。

英文摘要:

Metadata play a crucial role in ensuring the findability, accessibility, interoperability, and reusability of datasets. This paper investigates the potential of large language models (LLMs), specifically GPT-4, to improve adherence to metadata standards. We conducted experiments on 200 random data records describing human samples relating to lung cancer from the NCBI BioSample repository, evaluating GPT-4's ability to suggest edits for adherence to metadata standards. We computed the adherence accuracy of field name-field value pairs through a peer review process, and we observed a marginal average improvement in adherence to the standard data dictionary from 79% to 80% (p<0.5). We then prompted GPT-4 with domain information in the form of the textual descriptions of CEDAR templates and recorded a significant improvement to 97% from 79% (p<0.01). These results indicate that, while LLMs may not be able to correct legacy metadata to ensure satisfactory adherence to standards when unaided, they do show promise for use in automated metadata curation when integrated with a structured knowledge base

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑