LLM激活中幻觉的弱监督检测
Weakly Supervised Detection of Hallucinations in LLM Activations
AI总结：
提出基于子集扫描的弱监督审计方法，在无需预知模式类型或虚假陈述的情况下检测LLM激活中的幻觉及关键节点，性能媲美全监督分布外分类器。
摘要：
我们提出一种审计方法，用于识别大型语言模型（LLM）是否在其内部状态中编码了幻觉等模式，这些模式可能传播到下游任务。我们引入一种弱监督审计技术，采用子集扫描方法来检测来自预训练模型的LLM激活中的异常模式。重要的是，我们的方法不需要先验地知道模式类型。相反，它依赖于测试期间不含异常的参考数据集。此外，我们的方法能够识别负责编码这些模式的关键节点，这可能为微调特定子网络以缓解偏见提供重要见解。我们引入两种新的扫描方法，以处理异常句子的LLM激活，这些句子可能在任一方向上偏离预期分布。我们的结果证实了先前的发现，即BERT编码幻觉的内部能力有限，而OPT似乎能够在内部编码幻觉信息。重要的是，我们的扫描方法在事先未接触虚假陈述的情况下，表现可与完全监督的分布外分类器相媲美。
英文摘要：
We propose an auditing method to identify whether a large language model (LLM) encodes patterns such as hallucinations in its internal states, which may propagate to downstream tasks. We introduce a weakly supervised auditing technique using a subset scanning approach to detect anomalous patterns in LLM activations from pre-trained models. Importantly, our method does not need knowledge of the type of patterns a-priori. Instead, it relies on a reference dataset devoid of anomalies during testing. Further, our approach enables the identification of pivotal nodes responsible for encoding these patterns, which may offer crucial insights for fine-tuning specific sub-networks for bias mitigation. We introduce two new scanning methods to handle LLM activations for anomalous sentences that may deviate from the expected distribution in either direction. Our results confirm prior findings of BERT's limited internal capacity for encoding hallucinations, while OPT appears capable of encoding hallucination information internally. Importantly, our scanning approach, without prior exposure to false statements, performs comparably to a fully supervised out-of-distribution classifier.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
