当幻觉何时出现？从图视角看路径复用和路径压缩的演变
When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression
AI总结：
本文从图视角分析大语言模型中幻觉产生的机制，揭示路径复用与路径压缩对推理幻觉的影响，提出统一解释框架。
摘要：
大语言模型（LLMs）中的推理幻觉常常表现为流畅但缺乏支持的结论，这些结论违背了给定的上下文或底层事实知识。尽管此类失败现象广泛存在，但解码器-only Transformer 产生这些失败的机制仍不明确。我们把下一个词预测建模为对潜在图的图搜索过程，其中实体对应于节点，学习的转换形成边。从这个视角来看，上下文推理是受限于采样子图（内在推理）的搜索，而上下文无关查询则依赖于底层图中记忆化的结构（外在推理）。我们展示推理幻觉源于两种基本机制：路径复用，即在早期训练中记忆化的知识会覆盖上下文约束；路径压缩，即频繁遍历的多步路径在后期训练中会坍缩为快捷键边。这两种机制为LLMs中的推理幻觉提供了一个统一的解释，并与下游应用中观察到的已知行为相关联。
英文摘要：
Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. Although such failures are widely observed, the mechanisms by which decoder-only Transformers produce them remain poorly understood. We model next-token prediction as a graph search process over an underlying graph, where entities correspond to nodes and learned transitions form edges. From this perspective, contextual reasoning is a constrained search over a sampled subgraph (intrinsic reasoning), while context-free queries rely on memorized structures in the underlying graph (extrinsic reasoning). We show that reasoning hallucinations arise from two fundamental mechanisms: \textbf{Path Reuse}, where memorized knowledge overrides contextual constraints during early training, and \textbf{Path Compression}, where frequently traversed multi-step paths collapse into shortcut edges in later training. Together, these mechanisms provide a unified explanation for reasoning hallucinations in LLMs and connected to well-known behaviors observed in downstream applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
