通过动态认知协调解码缓解LLM中的上下文内存冲突
Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decoding
AI总结：
本文提出动态认知协调解码方法，通过分析注意力图预测潜在冲突并选择解码路径，提升LLM在处理上下文内存冲突时的准确性和效率，同时构建ConflictKG基准测试集验证效果。
摘要：
大型语言模型通过预训练积累大量参数知识。然而，当过时或错误的参数知识与上下文外部知识冲突时会出现知识冲突。现有方法通过对比解码处理冲突，但在无冲突场景中静态方法会破坏输出分布。其他动态解码方法尝试测量冲突程度但仍难以应对复杂现实情况。本文提出一种两阶段解码方法，称为动态认知协调解码（DCRD），用于预测和缓解上下文内存冲突。DCRD首先分析注意力图以评估上下文一致性并预测潜在冲突。基于此预测，输入被引导至两种解码路径之一：（1）贪婪解码，或（2）基于上下文一致性的动态解码。此设计使DCRD能够高效处理冲突，同时在无冲突情况下保持高准确性和解码效率。此外，为模拟频繁知识更新的场景，我们构建了ConflictKG，一个知识冲突问答基准测试集。在四个LLM上六个问答数据集上的实验表明，DCRD优于所有基线，达到最先进的性能。
英文摘要：
Large language models accumulate extensive parametric knowledge through pre-training. However, knowledge conflicts occur when outdated or incorrect parametric knowledge conflicts with external knowledge in the context. Existing methods address knowledge conflicts through contrastive decoding, but in conflict-free scenarios, static approaches disrupt output distribution. Other dynamic decoding methods attempt to measure the degree of conflict but still struggle with complex real-world situations. In this paper, we propose a two-stage decoding method called Dynamic Cognitive Reconciliation Decoding (DCRD), to predict and mitigate context-memory conflicts. DCRD first analyzes the attention map to assess context fidelity and predict potential conflicts. Based on this prediction, the input is directed to one of two decoding paths: (1) greedy decoding, or (2) context fidelity-based dynamic decoding. This design enables DCRD to handle conflicts efficiently while maintaining high accuracy and decoding efficiency in conflict-free cases. Additionally, to simulate scenarios with frequent knowledge updates, we constructed ConflictKG, a knowledge conflict QA benchmark. Experiments on four LLMs across six QA datasets show that DCRD outperforms all baselines, achieving state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
