Odysseus Navigates the Sirens' Song：面向事实性与多样性开放式文本生成的动态聚焦解码
Odysseus Navigates the Sirens' Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation
AI总结：
针对LLM开放式生成中事实性与多样性难以平衡的问题，提出即插即用的动态聚焦解码方法DFD，利用跨层分布差异自适应调整解码焦点，无需额外数据即可显著提升文本生成的事实性与多样性。
摘要：
大型语言模型（LLMs）在各种开放式应用中越来越需要生成既符合事实又具多样性的文本。然而，当前的随机解码方法难以平衡这些目标。我们提出了动态聚焦解码（DFD），这是一种新颖的即插即用随机方法，无需额外数据、知识或模型即可解决此权衡问题。DFD 利用 LLMs 内事实知识的模块化和分层特性，基于跨层的分布差异自适应地调整解码焦点。这种动态调整提升了知识密集型解码步骤中的事实性，并促进了较少依赖知识的步骤中的多样性。DFD 可轻松与现有解码方法集成，以最小的计算开销增强事实性与多样性。在七个数据集上的广泛实验表明，DFD 显著提升了性能，为开放式文本生成提供了可扩展且高效的解决方案。
英文摘要：
Large Language Models (LLMs) are increasingly required to generate text that is both factually accurate and diverse across various open-ended applications. However, current stochastic decoding methods struggle to balance such objectives. We introduce Dynamic Focus Decoding (DFD), a novel plug-and-play stochastic approach that resolves this trade-off without requiring additional data, knowledge, or models. DFD adaptively adjusts the decoding focus based on distributional differences across layers, leveraging the modular and hierarchical nature of factual knowledge within LLMs. This dynamic adjustment improves factuality in knowledge-intensive decoding steps and promotes diversity in less knowledge-reliant steps. DFD can be easily integrated with existing decoding methods, enhancing both factuality and diversity with minimal computational overhead. Extensive experiments across seven datasets demonstrate that DFD significantly improves performance, providing a scalable and efficient solution for open-ended text generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
