arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.01714cs.LGcs.CL
收藏

基于熵对齐的语言模型解码以提升写作与推理能力

Entropy-Aligned Decoding of LMs for Better Writing and Reasoning

Department of Computer Science University of California, Irvine(计算机科学系加州大学伊维特分校)

Kareem Ahmed, Sameer Singh

AI总结:

EPIC通过将未来轨迹的熵纳入语言模型解码,提升写作与推理能力,生成更多样且忠实的内容。

摘要:

语言模型(LMs)是在数十亿个标记上进行训练,试图恢复真实的语言分布。然而,普通的从LMs中随机采样生成的内容质量较低。解码算法试图将LM分布限制在一组高概率的延续上,但依赖于贪心启发式方法,引入了短视的扭曲,导致生成的句子同质、重复且不连贯。在本文中,我们引入了EPIC,一种无超参数的解码方法,该方法将未来轨迹的熵纳入LM解码中。EPIC明确地调节在生成过程中每一步所表达的不确定性程度,使采样分布的熵与数据的熵对齐。通过熵感知的懒惰Gumbel-Max采样,EPIC能够做到精确且高效,仅需每个步骤亚线性数量的熵评估。与当前基线不同,EPIC生成的采样分布在经验上与底层数据分布的熵对齐。在创造性写作和总结任务中,EPIC在广泛使用的解码策略上持续提高了LM-as-judge偏好胜率。这些偏好优势得到了自动指标的补充,显示EPIC生成的更多样化的内容和更忠实的总结。我们还评估了EPIC在数学推理中的表现,其在所有基线中均表现更优。

英文摘要:

Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM distribution to a set of high-probability continuations, but rely on greedy heuristics that introduce myopic distortions, yielding sentences that are homogeneous, repetitive and incoherent. In this paper, we introduce EPIC, a hyperparameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding. EPIC explicitly regulates the amount of uncertainty expressed at every step of generation, aligning the sampling distribution's entropy to the aleatoric (data) uncertainty. Through Entropy-Aware Lazy Gumbel-Max sampling, EPIC manages to be exact, while also being efficient, requiring only a sublinear number of entropy evaluations per step. Unlike current baselines, EPIC yields sampling distributions that are empirically well-aligned with the entropy of the underlying data distribution. Across creative writing and summarization tasks, EPIC consistently improves LM-as-judge preference win-rates over widely used decoding strategies. These preference gains are complemented by automatic metrics, showing that EPIC produces more diverse generations and more faithful summaries. We also evaluate EPIC on mathematical reasoning, where it outperforms all baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑