arXiv 2503.158882025-03-20cs.CLcs.AI

参数与上下文：语言模型中知识依赖的细粒度控制

Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models

University of Chinese Academy of Sciences（中国科学院大学）；University of California, Merced（加州大学默塞德分校）；National University of Singapore（新加坡国立大学）

Baolong Bi, Shenghua Liu, Yiwei Wang, Yilong Xu, Junfeng Fang, Lingrui Mei, Xueqi Cheng

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型中参数化知识与检索上下文的冲突问题，提出即插即用方法CK-PLUG，通过置信度增益检测冲突并调节token概率分布，实现知识依赖的细粒度控制，在反事实及通用RAG任务中均取得稳定性能提升。

摘要：

检索增强生成（Retrieval-Augmented Generation, RAG）通过整合外部知识缓解大语言模型（Large Language Models, LLMs）的幻觉问题。然而，参数化知识与检索到的上下文之间的冲突带来了挑战，尤其是当检索信息不可靠或模型内部知识过时的情况下。此时，LLMs难以判断应该更多依赖自身参数还是存在冲突的上下文。为解决这一问题，我们提出**CK-PLUG**，一种即插即用的方法，用于控制LLMs对参数化知识和上下文知识的依赖程度。我们提出了一种新颖的知识一致性指标——置信度增益（Confidence Gain），它通过测量上下文插入后token概率分布的熵变来检测知识冲突。随后，CK-PLUG通过单个调优参数调整具有负置信度增益的token的概率分布，实现对知识偏好的细粒度控制。实验表明，CK-PLUG能够在反事实RAG场景中显著调节知识依赖，同时保持生成流畅性和知识准确性。例如，在Llama3-8B上，RAG响应的记忆召回率（MR）可在9.9%-71.9%的大范围内调节，而基线为42.1%。此外，CK-PLUG支持基于模型对内部和外部知识的置信度进行自适应控制，在各类通用RAG任务中均取得了稳定的性能提升。我们的代码可通过以下链接获取：$\text{this https URL}$

英文摘要：

Retrieval-Augmented Generation (RAG) mitigates hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, conflicts between parametric knowledge and retrieved context pose challenges, particularly when retrieved information is unreliable or the model's internal knowledge is outdated. In such cases, LLMs struggle to determine whether to rely more on their own parameters or the conflicted context. To address this, we propose **CK-PLUG**, a plug-and-play method for controlling LLMs' reliance on parametric and contextual knowledge. We introduce a novel knowledge consistency metric, Confidence Gain, which detects knowledge conflicts by measuring entropy shifts in token probability distributions after context insertion. CK-PLUG then enables fine-grained control over knowledge preference by adjusting the probability distribution of tokens with negative confidence gain through a single tuning parameter. Experiments demonstrate CK-PLUG's ability to significantly regulate knowledge reliance in counterfactual RAG scenarios while maintaining generation fluency and knowledge accuracy. For instance, on Llama3-8B, memory recall (MR) of RAG response can be adjusted within a broad range (9.9%-71.9%), compared to the baseline of 42.1%. Moreover, CK-PLUG supports adaptive control based on the model's confidence in both internal and external knowledge, achieving consistent performance improvements across various general RAG tasks. Our code is available at: $\href{https://github.com/byronBBL/CK-PLUG}{\text{this https URL}}$.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。