CoRect: 用于隐藏状态校正的上下文感知logit对比以解决知识冲突
CoRect: Context-Aware Logit Contrast for Hidden State Rectification to Resolve Knowledge Conflicts
AI总结：
CoRect通过上下文感知logit对比校正隐藏状态，解决模型内部参数化知识与检索证据之间的冲突，提升生成任务的忠实度和减少幻觉。
摘要：
检索增强生成（RAG）经常在知识冲突中遇到困难，其中模型内部的参数化知识会覆盖检索到的证据，导致不忠实的输出。现有方法往往受限，要么依赖于表面解码调整，要么依赖于需要真实目标权重编辑。通过逐层分析，我们将这种失败归因于参数抑制现象：具体来说，在深层中，某些前馈网络（FFN）层会用记忆化的先验覆盖上下文敏感的表示。为了解决这个问题，我们提出了CoRect（用于隐藏状态校正的上下文感知logit对比）。通过对比上下文化和非上下文化的正向传递的logit，CoRect识别出表现出高参数偏置的层，而无需真实标签。然后它会校正隐藏状态以保留基于证据的信息。在问答（QA）和总结基准上，CoRect在忠实性和减少幻觉方面均优于强基线。
英文摘要：
Retrieval-Augmented Generation (RAG) often struggles with knowledge conflicts, where model-internal parametric knowledge overrides retrieved evidence, leading to unfaithful outputs. Existing approaches are often limited, relying either on superficial decoding adjustments or weight editing that necessitates ground-truth targets. Through layer-wise analysis, we attribute this failure to a parametric suppression phenomenon: specifically, in deep layers, certain FFN layers overwrite context-sensitive representations with memorized priors. To address this, we propose CoRect (Context-Aware Logit Contrast for Hidden State Rectification). By contrasting logits from contextualized and non-contextualized forward passes, CoRect identifies layers that exhibit high parametric bias without requiring ground-truth labels. It then rectifies the hidden states to preserve evidence-grounded information. Across question answering (QA) and summarization benchmarks, CoRect consistently improves faithfulness and reduces hallucinations compared to strong baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
