通过提示引导的数据增强增强幻觉检测
Bolster Hallucination Detection via Prompt-Guided Data Augmentation
AI总结：
针对幻觉检测标注数据稀缺问题，本文提出PALE框架，用提示引导LLM生成真实与幻觉样本，并设计CM Score检测稀疏嵌入，无需额外人工标注且性能较基线提升6.55%。
摘要：
大语言模型（LLM）在人工智能领域引起了广泛关注。尽管它们具有出色的生成能力，但研究发现它们会产生误导性或捏造的信息，这种现象被称为幻觉。因此，幻觉检测已成为确保LLM生成内容可靠性的关键。幻觉检测的一个主要挑战是缺乏同时包含真实输出和幻觉输出的、标注良好的数据集。为了解决这一问题，我们提出了提示引导数据增强幻觉检测（Prompt-guided data Augmented haLlucination dEtection，PALE），这是一个新颖框架，利用LLM在提示引导下产生的响应作为幻觉检测的数据增强。该策略能够在提示引导下以相对较低的成本生成真实数据和幻觉数据。为了更有效地评估LLM产生的稀疏中间嵌入的真实性，我们引入了一种称为对比马氏距离分数（Contrastive Mahalanobis Score，CM Score）的估计指标。该分数基于在激活空间中对真实数据和幻觉数据的分布进行建模。CM Score采用矩阵分解方法，以更准确地捕捉这些分布的底层结构。重要的是，我们的框架不需要额外的人工标注，为实际应用提供了很强的泛化能力和实用性。大量实验表明，PALE取得了优异的幻觉检测性能，以6.55%的显著优势优于具有竞争力的基线。
英文摘要：
Large language models (LLMs) have garnered significant interest in AI community. Despite their impressive generation capabilities, they have been found to produce misleading or fabricated information, a phenomenon known as hallucinations. Consequently, hallucination detection has become critical to ensure the reliability of LLM-generated content. One primary challenge in hallucination detection is the scarcity of well-labeled datasets containing both truthful and hallucinated outputs. To address this issue, we introduce Prompt-guided data Augmented haLlucination dEtection (PALE), a novel framework that leverages prompt-guided responses from LLMs as data augmentation for hallucination detection. This strategy can generate both truthful and hallucinated data under prompt guidance at a relatively low cost. To more effectively evaluate the truthfulness of the sparse intermediate embeddings produced by LLMs, we introduce an estimation metric called the Contrastive Mahalanobis Score (CM Score). This score is based on modeling the distributions of truthful and hallucinated data in the activation space. CM Score employs a matrix decomposition approach to more accurately capture the underlying structure of these distributions. Importantly, our framework does not require additional human annotations, offering strong generalizability and practicality for real-world applications. Extensive experiments demonstrate that PALE achieves superior hallucination detection performance, outperforming the competitive baseline by a significant margin of 6.55%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
