APCD：自适应路径对比解码用于可靠的大型语言模型生成
APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation
AI总结：
本文提出了一种自适应路径对比解码方法，通过自适应探索和受控路径交互提高大型语言模型生成的可靠性，实验表明在保持解码效率的同时提升了事实准确性。
摘要：
大型语言模型（LLMs）常常由于自回归解码中误差累积而产生幻觉，其中次优的早期token选择会误导后续生成。尽管多路径解码通过探索替代轨迹来提高鲁棒性，但现有方法缺乏确定何时分支和如何调节路径交互的系统策略。我们提出自适应路径对比解码（APCD），一种多路径解码框架，通过自适应探索和受控路径交互提高输出可靠性。APCD包含两个组成部分：（1）熵驱动路径扩展，延迟分支直到预测不确定性-通过香农熵测量在顶级候选token上的不确定性-表明存在多个可能的延续；以及（2）发散意识路径对比，鼓励多样化的推理轨迹同时动态减弱路径间的影响，随着预测分布发散。在八个基准测试中的实验显示，在保持解码效率的同时提高了事实准确性。我们的代码可在https://github.com/zty-king/APCD上获得。
英文摘要：
Reliable text generation is critical for deploying large language models (LLMs) in real-world applications, particularly in high-stakes domains such as medicine. To improve factual reliability, various inference-time methods have been proposed, including logit-level methods that modify token probability distributions and representation-level methods that manipulate intermediate model representations. However, most existing approaches operate on a single decoding trajectory, limiting their ability to explore alternative reasoning paths and making them susceptible to error accumulation. To address this limitation, we propose Adaptive Path-Contrastive Decoding (APCD), an adaptive multi-path contrastive decoding framework that improves factual reliability without model retraining or fine-tuning. APCD comprises two key components: Entropy-Driven Path Expansion, which adaptively expands the decoding process only at high-uncertainty decision points, and Divergence-Aware Path Contrast, which dynamically regulates contrastive interactions among parallel decoding paths based on their distributional divergence to balance diversity and coherence. We evaluate APCD on four LLM backbones across eight benchmarks spanning both general-domain and medical question answering tasks. Experimental results demonstrate that APCD consistently outperforms strong inference-time baselines in factual accuracy while maintaining competitive inference efficiency. These results demonstrate the robustness and generalizability of APCD across diverse models and tasks, highlighting its effectiveness as a practical multi-path decoding framework for reliable LLM deployment, particularly in high-stakes domains such as medicine. Code is available at https://github.com/zty-king/APCD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
