分层最优传输用于神经机器翻译和抽象摘要中的幻觉检测
Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization
AI总结：
通过最优传输分析跨注意力分布，发现幻觉检测集中于解码器前四层，且该方法在源脱离时有效，但无法检测注意力下游的不忠实摘要。
摘要：
最优传输（OT）已被证明可以通过测量跨注意力分布与参考分布之间的几何距离来检测神经机器翻译（NMT）中的幻觉，无需任何监督。我们将此分析扩展到Fairseq DE-EN模型的所有六个解码器层（$N=3{,}414$），表明Wass-to-Unif和Wass-to-Data是互补的检测器，专门针对不同类型的幻觉；检测集中在L1--L4层，而L5层对较微妙的类型具有反预测性；并且幻觉翻译缺乏正确翻译从第一步解码开始就存在的探索性注意力阶段。我们进一步评估了几何信号是否可迁移到抽象摘要忠实性检测：在AggreFact（$N=1{,}116$）上，我们的无监督OT检测器在CNN/XSum上达到$57.2\%$/$57.6\%$的平衡准确率——高于随机水平，但远低于有监督的MiniCheck-Flan-T5-L（$69.9\%$/$74.3\%$）。这种差距是原则性的：与NMT幻觉不同，不忠实的摘要可以正确关注源标记，同时歪曲其内容，这种失败模式在基于集中度的OT指标中由于构造原因而不可见。在T5-base上的结构实验证实了解码器在深度上的一致组织，其中第3层显示峰值集中度，第12层对生成质量最为关键。总之，结果确立了当失败模式是源脱离时，跨注意力的OT是一种可靠的检测器；无论任务如何，它都是一种原则性的可解释性工具；而当忠实性失败发生在注意力下游时，它则具有根本局限性。
英文摘要：
Optimal transport (OT) has been shown to detect hallucinations in neural machine translation (NMT) by measuring the geometric distance between cross-attention distributions and a reference distribution, without any supervision. We extend this analysis to all six decoder layers of the Fairseq DE-EN model ($N=3{,}414$), showing that Wass-to-Unif and Wass-to-Data are complementary detectors specialised across hallucination types, that detection is concentrated in layers L1--L4 with L5 anti-predictive for subtler types, and that hallucinated translations lack the exploratory attention phase present in correct translations from the first decoding step. We further evaluate whether the geometric signal transfers to abstractive summarization faithfulness detection: our unsupervised OT detector on AggreFact ($N=1{,}116$) achieves $57.2\%$/$57.6\%$ balanced accuracy on CNN/XSum -- above chance but substantially below supervised MiniCheck-Flan-T5-L($69.9\%$/$74.3\%$). This gap is principled: unlike NMT hallucinations, unfaithful summaries can attend correctly to source tokens while misrepresenting their content, a failure mode invisible to concentration-based OT metrics by construction. Structural experiments on T5-base confirm consistent decoder organisation across depth, with Layer~3 showing peak concentration and Layer~12 being most critical for generation quality. Together, the results establish OT on cross-attention as a reliable detector when the failure mode is source disengagement, a principled interpretability tool regardless of task, and fundamentally limited when faithfulness failures occur downstream of attention.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
