KITLM：面向问答的领域特定知识集成到语言模型
KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering
AI总结：
本文提出KITLM方法，通过向语言模型注入领域相关知识，在降低模型规模需求的同时显著提升问答性能，在MetaQA和AeroQA上均超越GPT-3.5-turbo及SKILL，并发布航空领域新基准AeroQA和Aviation Corpus数据集。
摘要：
大型语言模型（LLMs）在广泛的自然语言任务中展现了卓越的性能。然而，随着这些模型规模持续增长，它们在计算成本方面面临显著挑战。此外，LLMs通常缺乏高效的领域特定理解能力，这在航空和医疗等专业领域中尤为关键。为了提升领域特定理解能力，我们提出了KITLM，一种通过相关信息注入将知识库集成到语言模型中的新方法。通过集成相关知识，不仅语言模型的性能得到大幅提升，而且在实现可比性能的同时，模型规模需求也显著降低。我们提出的知识注入模型在性能上超越了GPT-3.5-turbo和当前最先进的知识注入方法SKILL，在MetaQA上的精确匹配分数实现了超过1.5倍的提升。KITLM在航空领域使用AeroQA时也展现了类似的性能提升。KITLM相较于现有方法的显著性能提升可归因于在注入相关知识的同时抑制了噪声。此外，我们发布了两个精心构建的数据集，以加速专业领域的知识注入研究：a) AeroQA，一个专为航空领域多跳问答设计的新基准数据集；b) Aviation Corpus，一个由从美国国家运输安全委员会报告中提取的非结构化文本构建的数据集。我们的研究有助于推动领域特定语言理解领域的发展，并展示了知识注入技术在提升语言模型问答性能方面的潜力。
英文摘要：
Large language models (LLMs) have demonstrated remarkable performance in a wide range of natural language tasks. However, as these models continue to grow in size, they face significant challenges in terms of computational costs. Additionally, LLMs often lack efficient domain-specific understanding, which is particularly crucial in specialized fields such as aviation and healthcare. To boost the domain-specific understanding, we propose, KITLM, a novel knowledge base integration approach into language model through relevant information infusion. By integrating pertinent knowledge, not only the performance of the language model is greatly enhanced, but the model size requirement is also significantly reduced while achieving comparable performance. Our proposed knowledge-infused model surpasses the performance of both GPT-3.5-turbo and the state-of-the-art knowledge infusion method, SKILL, achieving over 1.5 times improvement in exact match scores on the MetaQA. KITLM showed a similar performance boost in the aviation domain with AeroQA. The drastic performance improvement of KITLM over the existing methods can be attributed to the infusion of relevant knowledge while mitigating noise. In addition, we release two curated datasets to accelerate knowledge infusion research in specialized fields: a) AeroQA, a new benchmark dataset designed for multi-hop question-answering within the aviation domain, and b) Aviation Corpus, a dataset constructed from unstructured text extracted from the National Transportation Safety Board reports. Our research contributes to advancing the field of domain-specific language understanding and showcases the potential of knowledge infusion techniques in improving the performance of language models on question-answering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
