用于大语言模型忠实性幻觉与不对齐检测的提示-响应语义分歧指标
Prompt-Response Semantic Divergence Metrics for Faithfulness Hallucination and Misalignment Detection in Large Language Models
AI总结：
本文提出轻量级框架SDM，通过联合聚类与信息论指标测量提示与响应间的语义分歧，以检测大语言模型中的忠实性幻觉与不对齐问题，并构建诊断框架语义盒对响应类型进行分类。
摘要：
大型语言模型（LLMs）的普及面临着幻觉的挑战，这是一种关键的失效模式，模型会生成非事实、无意义或不忠实的文本。本文提出了语义分歧指标（SDM），这是一种用于检测忠实性幻觉的新型轻量级框架——忠实性幻觉是指LLM响应与输入上下文发生严重偏离的事件。我们关注这些LLM错误的一种具体实现，即虚构，定义为任意且与用户查询语义不对齐的响应。现有方法如语义熵通过测量对单一固定提示的答案多样性来测试任意性。我们的SDM框架对此进行了改进，更具提示感知能力：我们通过不仅测量多个答案间的响应一致性，还测量原始提示的多个语义等价释义间的响应一致性，来测试更深层次的任意性。在方法上，我们的方法对句子嵌入使用联合聚类，为提示和答案创建共享主题空间。提示和响应之间的主题共现热图可被视为用户-机器对话的量化二维可视化。然后，我们计算一套信息论指标来测量提示和响应之间的语义分歧。我们的实际分数 $\mathcal{S}_H$ 结合了Jensen-Shannon散度和Wasserstein距离来量化这种分歧，高分表示存在忠实性幻觉。此外，我们发现KL散度 KL(Answer $||$ Prompt) 是语义探索的强大指标，这是区分不同生成行为的关键信号。这些指标进一步组合成语义盒，这是一个用于分类LLM响应类型的诊断框架，包括危险的、自信的虚构。
英文摘要：
The proliferation of Large Language Models (LLMs) is challenged by hallucinations, critical failure modes where models generate non-factual, nonsensical or unfaithful text. This paper introduces Semantic Divergence Metrics (SDM), a novel lightweight framework for detecting Faithfulness Hallucinations -- events of severe deviations of LLMs responses from input contexts. We focus on a specific implementation of these LLM errors, {confabulations, defined as responses that are arbitrary and semantically misaligned with the user's query. Existing methods like Semantic Entropy test for arbitrariness by measuring the diversity of answers to a single, fixed prompt. Our SDM framework improves upon this by being more prompt-aware: we test for a deeper form of arbitrariness by measuring response consistency not only across multiple answers but also across multiple, semantically-equivalent paraphrases of the original prompt. Methodologically, our approach uses joint clustering on sentence embeddings to create a shared topic space for prompts and answers. A heatmap of topic co-occurances between prompts and responses can be viewed as a quantified two-dimensional visualization of the user-machine dialogue. We then compute a suite of information-theoretic metrics to measure the semantic divergence between prompts and responses. Our practical score, $\mathcal{S}_H$, combines the Jensen-Shannon divergence and Wasserstein distance to quantify this divergence, with a high score indicating a Faithfulness hallucination. Furthermore, we identify the KL divergence KL(Answer $||$ Prompt) as a powerful indicator of \textbf{Semantic Exploration}, a key signal for distinguishing different generative behaviors. These metrics are further combined into the Semantic Box, a diagnostic framework for classifying LLM response types, including the dangerous, confident confabulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
