打破大型语言模型忠实性与表现力之间的权衡
Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models
AI总结：
提出协同解码（CoDe）框架，通过动态整合有无外部知识的输出概率并引入知识感知重排序，打破LLM忠实性与表现力之间的权衡，实验证明其有效性和泛化性。
摘要：
将回答基于外部知识是缓解大型语言模型（LLMs）幻觉的有效策略。然而，当前的LLMs难以在保持忠实性（或保真度）和表现力的同时无缝整合知识，而这两者是人类自然具备的能力。这一局限导致输出要么缺乏外部知识的支持，从而损害忠实性；要么显得过于冗长和不自然，从而牺牲表现力。在本工作中，为了打破忠实性与表现力之间的权衡，我们提出了协同解码（CoDe），一种新颖的方法，它动态整合有无外部知识时生成的输出概率。这种整合由分布差异和模型置信度引导，使得能够从模型内部参数中选择性激活相关且可靠的表达。此外，我们引入了一种知识感知的重新排序机制，以防止过度依赖先验参数知识，同时确保适当利用提供的外部信息。通过全面的实验，我们的即插即用CoDe框架在增强忠实性而不损害表现力方面，在多种LLMs和评估指标上展现出优越性能，验证了其有效性和泛化性。
英文摘要：
Grounding responses in external knowledge represents an effective strategy for mitigating hallucinations in Large Language Models (LLMs). However, current LLMs struggle to seamlessly integrate knowledge while simultaneously maintaining faithfulness (or fidelity) and expressiveness, capabilities that humans naturally possess. This limitation results in outputs that either lack support from external knowledge, thereby compromising faithfulness, or appear overly verbose and unnatural, thus sacrificing expressiveness. In this work, to break the trade-off between faithfulness and expressiveness, we propose Collaborative Decoding (CoDe), a novel approach that dynamically integrates output probabilities generated with and without external knowledge. This integration is guided by distribution divergence and model confidence, enabling the selective activation of relevant and reliable expressions from the model's internal parameters. Furthermore, we introduce a knowledge-aware reranking mechanism that prevents over-reliance on prior parametric knowledge while ensuring proper utilization of provided external information. Through comprehensive experiments, our plug-and-play CoDe framework demonstrates superior performance in enhancing faithfulness without compromising expressiveness across diverse LLMs and evaluation metrics, validating both its effectiveness and generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
