arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.04844cs.CL
收藏

Osiris:一个轻量级开源幻觉检测系统

Osiris: A Lightweight Open-Source Hallucination Detection System

Stanford University(斯坦福大学);Stanford HAI(斯坦福HAI)

Alex Shan, John Bauer, Christopher D. Manning

AI总结:

本文提出轻量级开源幻觉检测系统Osiris,通过构建带诱导幻觉的扰动多跳问答数据集并微调7B模型,在RAGTruth基准上以更少参数超越GPT-4o的召回率,为RAG系统幻觉检测提供低成本可扩展方案。

摘要:

检索增强生成(RAG)系统已被应用开发者广泛采用,因为它们利用事实来源使大语言模型(LLM)能够生成在事实上更可靠的回答。然而,幻觉——即LLM回答不忠实于所提供上下文的实例——常常阻碍这些系统在生产环境中部署。当前的幻觉检测方法通常涉及人工评估或使用闭源模型来审查RAG系统输出中的幻觉。人工评估者和闭源模型都因高成本和较慢的推理速度而面临扩展性问题。在这项工作中,我们引入了一个带有诱导幻觉的扰动多跳问答数据集。通过在我们的数据集上进行监督微调,我们使用一个7B模型在RAGTruth幻觉检测基准上取得了比GPT-4o更好的召回率,并在精确率和准确率上提供了有竞争力的表现,同时仅使用了极少一部分参数。代码已发布在我们的代码仓库中。

英文摘要:

Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance on precision and accuracy, all while using a fraction of the parameters. Code is released at our repository.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑