CoCoA：用于解决大语言模型中知识冲突的置信度与上下文感知自适应解码
CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models
AI总结：
本文提出CoCoA，一种置信度与上下文感知的自适应解码算法，通过熵差、上下文峰值度和广义散度解决大语言模型的知识冲突，在问答、摘要和长文本问答任务上显著超越AdaCAD等基线，提升准确率与事实性。
摘要：
大语言模型（LLMs）中的忠实生成受到参数化记忆与外部上下文之间知识冲突的挑战。现有专门针对冲突处理的对比解码方法往往缺乏适应性，并且在低冲突场景下性能会下降。我们提出了CoCoA（Confidence- and Context-Aware Adaptive Decoding，置信度与上下文感知自适应解码），这是一种新颖的token级算法，用于有原则地解决冲突并增强忠实性。CoCoA通过利用置信度感知度量（熵差和上下文峰值度）以及参数化分布与上下文分布之间的广义散度来解决冲突。关键的是，CoCoA即使在低冲突场景下也能保持强劲性能。在多个大语言模型上，针对多样化的问答（QA）、摘要和长文本问答（LFQA）基准进行的大量实验表明，CoCoA相较于AdaCAD等强基线达到了最先进的性能。与强基线AdaCAD相比，它在问答准确率上平均提升高达9.2个百分点，并在关键基准上将摘要和长文本问答的事实性平均提升高达2.5个百分点。此外，它还展示了对冲突变化的卓越敏感性。CoCoA实现了更明智、更具上下文感知且最终更忠实的token生成。
英文摘要：
Faithful generation in large language models (LLMs) is challenged by knowledge conflicts between parametric memory and external context. Existing contrastive decoding methods tuned specifically to handle conflict often lack adaptability and can degrade performance in low conflict settings. We introduce CoCoA (Confidence- and Context-Aware Adaptive Decoding), a novel token-level algorithm for principled conflict resolution and enhanced faithfulness. CoCoA resolves conflict by utilizing confidence-aware measures (entropy gap and contextual peakedness) and the generalized divergence between the parametric and contextual distributions. Crucially, CoCoA maintains strong performance even in low conflict settings. Extensive experiments across multiple LLMs on diverse Question Answering (QA), Summarization, and Long-Form Question Answering (LFQA) benchmarks demonstrate CoCoA's state-of-the-art performance over strong baselines like AdaCAD. It yields significant gains in QA accuracy, up to 9.2 points on average compared to the strong baseline AdaCAD, and improves factuality in summarization and LFQA by up to 2.5 points on average across key benchmarks. Additionally, it demonstrates superior sensitivity to conflict variations. CoCoA enables more informed, context-aware, and ultimately more faithful token generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
