arXiv 2604.062772026-04-07cs.AIcs.CLcs.LG

弱监督蒸馏 hallucination 信号到 Transformer 表示

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

LLM Lens

Shoaib Sadiq Salehmohamed, Jinal Prashant Thakkar, Hansika Aredla, Shaik Mohammed Omar, Shalmali Ayachit

备注：: 20 pages, 6 figures, 6 tables. Introduces a 15k-sample representation-level hallucination dataset with full transformer hidden states and multi-signal weak supervision. Evaluates 5 probing architectures and demonstrates internal hallucination detection without external inference-time signals. Includes held-out test evaluation and deployment benchmarks

AI总结：

本文提出弱监督框架，通过子串匹配、句子嵌入相似性和LLM判官信号，无需人工标注即可标注生成响应是否 grounded 或 hallucinated，并通过五种探针分类器验证 hallucination 信号可被蒸馏到 Transformer 表示中。

摘要：

现有大型语言模型（LLM）的hallucination检测方法依赖于推理时的外部验证，需要金答案、检索系统或辅助判断模型。我们问这种外部监督是否能被蒸馏到模型自身表示中进行训练，从而在推理时仅通过内部激活进行hallucination检测。我们引入了一个弱监督框架，结合三种互补的grounding信号：子串匹配、句子嵌入相似性和LLM作为判断结果，以标注生成响应是否为grounded或hallucinated，而无需人工标注。使用此框架，我们从SQuAD v2构建了一个15000样本的数据集（10500训练/开发样本和一个独立的5000样本测试集），其中每个示例配对了一个LLaMA-2-7B生成的答案及其完整的每层隐藏状态和结构化hallucination标签。然后我们训练了五个探针分类器：ProbeMLP（M0）、LayerWiseMLP（M1）、CrossLayerTransformer（M2）、HierarchicalTransformer（M3）和CrossLayerAttentionTransformerV2（M4），直接在这些隐藏状态上进行训练，将外部grounding信号仅作为训练时的监督。我们的核心假设是hallucination检测信号可以被蒸馏到transformer表示中，从而在推理时无需任何外部验证。结果支持这一假设。基于transformer的探针在最强的判别能力，其中M2在5折平均AUC/F1表现最佳，而M3在单折验证和保持测试评估中表现最佳。我们还进行了推理效率的基准测试：探针延迟范围从0.15到5.62毫秒（批量）和1.55到6.66毫秒（单样本），而端到端生成加探针吞吐量仍约为0.231查询每秒，表明实际开销 negligible。

英文摘要：

Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retrieval systems, or auxiliary judge models. We ask whether this external supervision can instead be distilled into the model's own representations during training, enabling hallucination detection from internal activations alone at inference time. We introduce a weak supervision framework that combines three complementary grounding signals: substring matching, sentence embedding similarity, and an LLM as a judge verdict to label generated responses as grounded or hallucinated without human annotation. Using this framework, we construct a 15000-sample dataset from SQuAD v2 (10500 train/development samples and a separate 5000-sample test set), where each example pairs a LLaMA-2-7B generated answer with its full per-layer hidden states and structured hallucination labels. We then train five probing classifiers: ProbeMLP (M0), LayerWiseMLP (M1), CrossLayerTransformer (M2), HierarchicalTransformer (M3), and CrossLayerAttentionTransformerV2 (M4), directly on these hidden states, treating external grounding signals as training-time supervision only. Our central hypothesis is that hallucination detection signals can be distilled into transformer representations, enabling internal detection without any external verification at inference time. Results support this hypothesis. Transformer-based probes achieve the strongest discrimination, with M2 performing best on 5-fold average AUC/F1, and M3 performing best on both single-fold validation and held-out test evaluation. We also benchmark inference efficiency: probe latency ranges from 0.15 to 5.62 ms (batched) and 1.55 to 6.66 ms (single sample), while end-to-end generation plus probe throughput remains approximately 0.231 queries per second, indicating negligible practical overhead.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。