发表机构
University of Bamberg（班贝格大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文发现网格束搜索在词汇约束解码中偏向先满足易约束，提出公平网格束搜索以消除偏差，同时保持线性前向传递并找到更高概率字符串。
AI 中文摘要
约束自回归文本生成模型输出的一种常见形式涉及词汇约束，即要求出现在生成文本中的单词或短语。DFA约束束搜索和网格束搜索是在强制执行词汇约束时从自回归模型解码的两种广泛使用的范式。由于前一种方法需要的前向传递次数随约束标记数量呈指数增长，因此通常不如后一种方法受欢迎，后者仅需要线性数量的前向调用。然而，虽然网格束搜索实现了指数级的加速，但它以一种不平等对待所有约束的方式实现。在本文中，我们证明网格束搜索偏向于首先满足较易满足的约束，将较难的约束留到序列末尾。这与DFA约束束搜索形成对比，后者没有这种偏差。为了解决这一缺陷，我们提出了公平网格束搜索，这是对网格束搜索的一种修改，避免了这种偏差，同时仍然仅需要线性数量的前向传递。在实验上，我们在两个受约束的生成任务上确认了网格束搜索的偏差，发现其约束标记排序方式与DFA约束束搜索和公平网格束搜索相比存在显著差异。此外，我们发现公平网格束搜索不仅修复了网格束搜索的偏差，而且在此过程中找到了更高概率的字符串。
英文摘要
A common formalism for constraining the output of autoregressive text generation models involves lexical constraints, words or phrases which are required to occur in the generated text. DFA-constrained beam search and grid beam search are two widely used paradigms for decoding from autoregressive models while enforcing lexical constraints. As the former approach requires a number of forward passes exponential in the number of constraint tokens, it is often dispreferred to the latter, which requires only linearly many forward calls. However, while grid beam search achieves an exponential speedup, it does so in a manner which does not treat all of the constraints equally. In this paper, we demonstrate that grid beam search is biased to incorporate easier-to-satisfy constraints first, leaving harder constraints to the end of the sequence. This contrasts with DFA-constrained beam search, which exhibits no such bias. To address this shortcoming, we propose fair grid beam search, a modification to grid beam search which avoids this bias while still requiring only linearly many forward passes. Experimentally, we confirm grid beam search's bias on two constrained generation tasks, finding significant differences in how it orders constraint tokens as compared to DFA-constrained beam search and fair grid beam search. Furthermore, we find that fair grid beam search not only fixes grid beam search's bias, but finds higher-probability strings in the process.
CommentsPublished as a conference paper at COLM 2026
Journal refProceedings of the Third Conference on Language Modeling (COLM 2026)
