arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.05385cs.CLcs.AI
收藏

面向构建具备大语言模型的鲁棒性知识密集型问答模型

Towards Building a Robust Knowledge Intensive Question Answering Model with Large Language Models

Zhejiang Lab(之江实验室);China Mobile, Hangzhou Research and Development Center(中国移动杭州研发中心)

Xingyun Hong, Yan Shao, Zhilin Wang, Manni Duan, Jin Xiongnan

AI总结:

针对检索信息噪声影响大语言模型问答鲁棒性的问题,构建多干扰场景数据集,提出数据增强微调结合对比学习的方法,经GPT-4评估可同时提升模型鲁棒性与信息判别能力。

摘要:

大语言模型(LLMs)的发展极大提升了问答的智能性与流畅度,检索增强技术的出现则让模型能更好地利用外部信息。然而,检索信息中存在的噪声与错误给LLMs的鲁棒性带来了挑战。本研究中,为评估模型在多重干扰下的性能,我们首先基于机器阅读理解数据集构建了一个模拟多种场景的数据集,涵盖关键信息缺失、噪声及冲突等情况。针对外部信息噪声导致的模型准确率下降问题,我们提出了一种基于数据增强的微调方法,以提升LLM的抗噪声鲁棒性。此外,我们还采用对比学习方法来保留模型对外部信息的判别能力。我们在现有LLMs和我们的方法上均开展了实验,由GPT-4进行评估的结果表明,我们提出的方法在提升模型鲁棒性的同时,也增强了模型的判别能力。

英文摘要:

The development of LLMs has greatly enhanced the intelligence and fluency of question answering, while the emergence of retrieval enhancement has enabled models to better utilize external information. However, the presence of noise and errors in retrieved information poses challenges to the robustness of LLMs. In this work, to evaluate the model's performance under multiple interferences, we first construct a dataset based on machine reading comprehension datasets simulating various scenarios, including critical information absence, noise, and conflicts. To address the issue of model accuracy decline caused by noisy external information, we propose a data augmentation-based fine-tuning method to enhance LLM's robustness against noise. Additionally, contrastive learning approach is utilized to preserve the model's discrimination capability of external information. We have conducted experiments on both existing LLMs and our approach, the results are evaluated by GPT-4, which indicates that our proposed methods improve model robustness while strengthening the model's discrimination capability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑