SLED：用于提升大型语言模型事实性的自对数概率演变解码
SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models
AI总结：
针对大型语言模型输出的事实错误问题，提出无需外部知识库或微调的SLED解码框架，通过对比不同层对数概率并利用近似梯度引导自我优化，在多模型基准上提升事实准确性且保持流畅性与低延迟。
摘要：
大型语言模型（LLMs）已展现出卓越能力，但其输出有时可能不可靠或存在事实错误。为此，我们提出自对数概率演变解码（SLED），这是一种无需依赖外部知识库或进一步微调即可增强LLMs真实性的新型解码框架。从优化角度看，SLED框架通过对比最终层与早期层的输出对数概率，利用LLM内嵌的潜在知识，再采用近似梯度方法让潜在知识引导输出的自我优化，从而有效提升事实准确性。我们在多种模型家族（Gemma、Qwen、Mixtral、gpt-oss）和规模（1B至45B）的现有基准上开展了大量实验，包括混合专家（MoE）等更先进的架构配置。评估覆盖多种任务，结果显示SLED在保持自然语言流畅性和可忽略的延迟开销的同时，相比现有解码方法持续提升事实准确性，且能灵活与其他解码方法结合进一步增强性能。
英文摘要：
Large language models (LLMs) have demonstrated remarkable capabilities, but their outputs can sometimes be unreliable or factually incorrect. To address this, we introduce Self Logits Evolution Decoding (SLED), a novel decoding framework that enhances the truthfulness of LLMs without relying on external knowledge bases or requiring further fine-tuning. From an optimization perspective, our SLED framework leverages the latent knowledge embedded within the LLM by contrasting the output logits from the final layer with those from early layers. It then utilizes an approximate gradient approach to enable latent knowledge to guide the self-refinement of outputs, thereby effectively improving factual accuracy. Extensive experiments have been conducted on established benchmarks across a diverse range of model families (Gemma, Qwen, Mixtral, gpt-oss) and scales (from 1B to 45B), including more advanced architectural configurations such as the mixture of experts (MoE). Our evaluation spans a wide variety of tasks and the results demonstrate that SLED consistently improves factual accuracy compared to existing decoding methods while maintaining natural language fluency and negligible latency overhead. Furthermore, it can be flexibly combined with other decoding methods to further enhance their performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
