HalluRAG数据集：利用大语言模型的内部状态检测RAG应用中的闭域幻觉
The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States
AI总结：
提出HalluRAG数据集，利用LLM内部状态在句子级别检测RAG应用中未见于训练数据的幻觉，MLP分类器测试准确率最高达75%，并发现IAV与CEV效果相当。
摘要：
检测大语言模型（LLM）中的幻觉对于提高其可靠性和可信度至关重要。大多数研究将幻觉视为与训练期间所见信息的偏差。然而，LLM参数化知识的不透明性使得理解生成文本为何看似缺乏依据变得复杂：LLM可能没有从庞大且往往难以访问的数据集中获取必要知识，或者信息在进一步训练过程中可能被改变或产生矛盾。我们关注的是涉及未在训练中使用的信息的幻觉，我们通过利用时效性来确保这些信息在截止日期之后出现，从而加以确定。本研究通过使用各种LLM的不同内部状态在句子级别检测这些幻觉来进行调查。我们提出了HalluRAG，一个旨在训练分类器检测这些幻觉的数据集。根据模型和量化方式的不同，在HalluRAG上训练的MLP在测试中的准确率最高可达75%，其中Mistral-7B-Instruct-v0.1取得了最高的测试准确率。我们的结果表明，IAVs检测幻觉的效果与CEVs相当，并揭示可回答和不可回答的提示以不同方式编码，因为针对这些类别的单独分类器提高了准确率。然而，HalluRAG显示出一定的泛化局限性，这主张在幻觉数据集方面需要更多多样性。
英文摘要：
Detecting hallucinations in large language models (LLMs) is critical for enhancing their reliability and trustworthiness. Most research focuses on hallucinations as deviations from information seen during training. However, the opaque nature of an LLM's parametric knowledge complicates the understanding of why generated texts appear ungrounded: The LLM might not have picked up the necessary knowledge from large and often inaccessible datasets, or the information might have been changed or contradicted during further training. Our focus is on hallucinations involving information not used in training, which we determine by using recency to ensure the information emerged after a cut-off date. This study investigates these hallucinations by detecting them at sentence level using different internal states of various LLMs. We present HalluRAG, a dataset designed to train classifiers on these hallucinations. Depending on the model and quantization, MLPs trained on HalluRAG detect hallucinations with test accuracies ranging up to 75 %, with Mistral-7B-Instruct-v0.1 achieving the highest test accuracies. Our results show that IAVs detect hallucinations as effectively as CEVs and reveal that answerable and unanswerable prompts are encoded differently as separate classifiers for these categories improved accuracy. However, HalluRAG showed some limited generalizability, advocating for more diversity in datasets on hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
