分布语义追踪：一种解释大语言模型幻觉的框架
Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models
AI总结：
本文提出分布语义追踪框架，通过解码残差流状态构建语义图谱，揭示大语言模型幻觉源于表征漂移，通过轻量因果追踪验证假设，实验显示其解释效果优于基线方法。
摘要：
大语言模型（LLMs）的幻觉会产生不被提示支持的流畅延续，尤其在最小上下文提示和歧义情况下更明显。本文引入分布语义追踪（DST），一种模型原生方法，通过解码残差流状态构建答案位置的分层语义图谱，选择紧凑的top-K概念集，并通过轻量因果追踪估计概念间的定向支持。利用这些追踪，我们测试了表征层面的假设：幻觉源于深度跨层的相关驱动表征漂移，其中残差流被训练共现关系拉向局部一致但上下文不一致的概念邻域。在Racing Thoughts数据集上，DST在LLM- judge协议下比归因、探测和干预基线更忠实，所得到的上下文对齐分数（CAS）强烈预测失败，支持这一漂移假设。
英文摘要：
Hallucinations in large language models (LLMs) produce fluent continuations that are not supported by the prompt, especially under minimal contextual cues and ambiguity. We introduce Distributional Semantics Tracing (DST), a model-native method that builds layer-wise semantic maps at the answer position by decoding residual-stream states through the unembedding, selecting a compact top-$K$ concept set, and estimating directed concept-to-concept support via lightweight causal tracing. Using these traces, we test a representation-level hypothesis: hallucinations arise from correlation-driven representational drift across depth, where the residual stream is pulled toward a locally coherent but context-inconsistent concept neighborhood reinforced by training co-occurrences. On Racing Thoughts dataset, DST yields more faithful explanations than attribution, probing, and intervention baselines under an LLM-judge protocol, and the resulting Contextual Alignment Score (CAS) strongly predicts failures, supporting this drift hypothesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
