Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM distribution to a set of high-probability continuations, but rely on greedy heuristics that introduce myopic distortions, yielding sentences that are homogeneous, repetitive and incoherent. In this paper, we introduce EPIC, a hyperparameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding. EPIC explicitly regulates the amount of uncertainty expressed at every step of generation, aligning the sampling distribution's entropy to the aleatoric (data) uncertainty. Through Entropy-Aware Lazy Gumbel-Max sampling, EPIC manages to be exact, while also being efficient, requiring only a sublinear number of entropy evaluations per step. Unlike current baselines, EPIC yields sampling distributions that are empirically well-aligned with the entropy of the underlying data distribution. Across creative writing and summarization tasks, EPIC consistently improves LM-as-judge preference win-rates over widely used decoding strategies. These preference gains are complemented by automatic metrics, showing that EPIC produces more diverse generations and more faithful summaries. We also evaluate EPIC on mathematical reasoning, where it outperforms all baselines.
TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques
Faculty of Electrical and Computer Engineering, University of Tabriz(大不里士大学电气与计算机工程学院);Faculty of Information Technology and Computer Engineering, Azarbaijan Shahid Madani University(阿扎尔巴伊詹沙希德马达尼大学信息技术与计算机工程学院)
An Analysis of the Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
Amazon Alexa AI(亚马逊Alexa人工智能部门);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Information Sciences Institute, University of Southern California(南加利福尼亚大学信息科学研究所)
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Stanford University(斯坦福大学)
Mitigating Errors in LLM-Generated Web API Invocations via Retrieval-Augmented Generation and Constrained Decoding
Technische Universität Darmstadt(德累斯顿技术大学);Hessian Center for Artificial Intelligence (hessian.AI)(黑森人工智能中心);Pariton AI;National Research Center for Applied Cybersecurity ATHENE(应用网络安全国家研究中心ATHENE)