TRACE: 通过跨层证据进行轨迹修正以减少幻觉
TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction
AI总结：
本文提出TRACE算法，通过跨层证据在推理时修正LLM中的幻觉，无需训练或标注，通过内部证据选择修正策略，提升多个基准测试的性能。
摘要：
幻觉修正并非单向问题。我们表明中间层既不总是比最终层更诚实，也不总是更不可信。然而，幻觉减少通常通过固定干预形式实现：对比一层另一层，引导沿诚实方向，或依赖外部证据。这种框架结构不完整。跨层事实证据并不均匀演变：在某些失败中，内部真实支持存在并随后被压制，而在其他情况下，候选竞争在深度上保持 genuinely 多方向性，因此没有单一符号标量家族通常足够。我们引入TRACE（Trajectory Correction from Cross-layer Evidence for Hallucination Reduction），一种确定性、无训练的算法，在LLM自身前向传递中，通过从每个输入的跨层候选轨迹中推导出修正层和适当的修正操作符，在推理时修正幻觉。在一种冻结超参数设置下，TRACE仅使用模型内部证据，在8个模型家族和3个事实性基准测试中评估为单一通用算法，对15个模型进行评估，提升所有评估单元，产生+12.26 MC1点和+8.65 MC2风格点的平均增益，无退化，增益达到+47.20 MC1和+43.38 MC2风格点。该方法不使用标签、检索、预训练、微调或每模型校准。
英文摘要：
Hallucination correction is not a one-direction problem. We show that intermediate layers are neither uniformly more truthful than final layers nor uniformly less trustworthy. Yet hallucination reduction is usually instantiated through one fixed intervention form: contrast one layer against another, steer along a truthfulness direction, or defer to external evidence. This framing is structurally incomplete. Cross-layer factual evidence does not evolve uniformly: in some failures truthful support is present internally and later suppressed, whereas in others candidate competition remains genuinely multi-directional across depth, so no single signed scalar family is generally sufficient. We introduce Trajectory Correction from Cross-layer Evidence for Hallucination Reduction (TRACE), a deterministic, training-free algorithm which corrects hallucinations at inference time by deriving both the corrective layer and the appropriate correction operator from each input's cross-layer candidate trajectory inside the LLM's own forward pass. Under one frozen hyperparameter setting, TRACE selects among scalar reversal, earlier-state recovery, and candidate-space correction using only model-internal evidence. Evaluated as a single universal algorithm across 15 models, 8 model families, and 3 factuality benchmarks, TRACE improves every evaluation cell, yielding mean gains of +12.26 MC1 points and +8.65 MC2-style points with no regressions, with gains reaching +47.20 MC1 and +43.38 MC2-style points. The method uses no labels, retrieval, pretraining, finetuning, or per-model calibration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
