REFIND 在 SemEval-2025 任务 3：大语言模型中检索增强的事实性幻觉检测
REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models
AI总结：
REFIND 提出利用检索文档和上下文敏感度比率 CSR 来检测大语言模型输出中的幻觉片段，在九种语言上显著优于基线模型。
摘要：
大语言模型（LLM）输出中的幻觉严重限制了其在问答等知识密集型任务中的可靠性。为应对这一挑战，我们提出 REFIND（Retrieval-augmented Factuality hallucINation Detection，检索增强的事实性幻觉检测），这是一个通过直接利用检索到的文档来检测 LLM 输出中幻觉片段的新型框架。作为 REFIND 的一部分，我们提出了上下文敏感度比率（Context Sensitivity Ratio，CSR），这是一种量化 LLM 输出对检索证据敏感度的新型指标。这一创新方法使 REFIND 能够高效且准确地检测幻觉，使其区别于现有方法。在评估中，REFIND 在包括低资源环境在内的九种语言上表现出鲁棒性，并显著优于基线模型，在识别幻觉片段方面取得了更优的 IoU 分数。本工作凸显了量化上下文敏感度在幻觉检测中的有效性，从而为跨多种语言构建更可靠、更可信的 LLM 应用铺平了道路。我们的代码可在 https://github.com/oneonlee/REFIND 获取。
英文摘要：
Hallucinations in large language model (LLM) outputs severely limit their reliability in knowledge-intensive tasks such as question answering. To address this challenge, we introduce REFIND (Retrieval-augmented Factuality hallucINation Detection), a novel framework that detects hallucinated spans within LLM outputs by directly leveraging retrieved documents. As part of the REFIND, we propose the Context Sensitivity Ratio (CSR), a novel metric that quantifies the sensitivity of LLM outputs to retrieved evidence. This innovative approach enables REFIND to efficiently and accurately detect hallucinations, setting it apart from existing methods. In the evaluation, REFIND demonstrated robustness across nine languages, including low-resource settings, and significantly outperformed baseline models, achieving superior IoU scores in identifying hallucinated spans. This work highlights the effectiveness of quantifying context sensitivity for hallucination detection, thereby paving the way for more reliable and trustworthy LLM applications across diverse languages. Our code is available at https://github.com/oneonlee/REFIND.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
