arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.16686cs.CLcs.AI
收藏

无劣上下文感知解码:防止上下文条件生成中的中性退化

No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation

Department of Computer Science, Portland State University, USA(波特兰州立大学计算机科学系)

Yufei Tao, Ameeta Agrawal

AI总结:

本文提出NWCAD解码器,通过双流架构和双阶段门控机制,防止中性退化并提升上下文利用的可靠性。

摘要:

大型语言模型(LLMs)在接收到外部上下文(如检索证据)时能够回答问题和总结文档,但上下文使用仍不可靠:模型可能在上下文非信息性时覆盖已正确输出(中性退化)。我们正式将中性退化定义为无害要求,并通过测量基线修正项目在答案一致上下文下的准确性下降来量化。我们提出无劣上下文感知解码(NWCAD),一种基于双流架构的解码时间适配器,包含双阶段门控:当上下文非信息性时退化到无上下文解码,否则使用上下文条件解码并在不确定时采用CAD式回退。我们在分离无害可靠性与上下文利用(在真正有益上下文中准确性提升)的基准上评估NWCAD。NWCAD在基线修正项目上防止中性退化,同时在有益上下文中保持强上下文驱动准确性。

英文摘要:

Large language models (LLMs) can answer questions and summarize documents when conditioned on external contexts (e.g., retrieved evidence), yet context use remains unreliable: models may overwrite an already-correct output (neutral regression) even when the context is non-informative. We formalize neutral regression as a do-no-harm requirement and quantify it by measuring accuracy drops on baseline-correct items under answer-consistent contexts. We propose No-Worse Context-Aware Decoding (NWCAD), a decode-time adapter built on a two-stream setup with a two-stage gate: it backs off to no-context decoding when the context is non-informative, and otherwise uses context-conditioned decoding with a CAD-style fallback under uncertainty. We evaluate NWCAD on benchmarks that separate do-no-harm reliability from context utilization (accuracy gains on genuinely helpful contexts). NWCAD prevents neutral regression on baseline-correct items while preserving strong context-driven accuracy on helpful contexts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑