arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.12018cs.CL
收藏

BOLT:具有可调偏置的快速基于能量的受控文本生成

BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases

University of Michigan(密歇根大学)

Xin Liu, Muhammad Khalifa, Lu Wang

AI总结:

提出BOLT方法,通过可调偏置直接调整语言模型输出logits并保持自回归特性,在受控文本生成任务中实现更快的收敛速度与更高的生成效率及流畅性。

摘要:

能量模型(EBMs)因其对广泛约束的高度适用性而在受控文本生成中受到欢迎。然而,从 EBMs 中采样并非易事,因为它通常需要大量迭代才能收敛到合理的文本,这减慢了解码过程并使其在现实应用中不太实用。在这项工作中,我们提出了 BOLT,它依赖于可调偏置来直接调整语言模型的输出 logits。与先前的工作不同,BOLT 保持了生成器的自回归特性,以对 token 级别的条件依赖性和整体流畅性施加强控制,从而收敛更快。在使用软约束(例如情感控制)和硬约束(例如关键词引导的主题控制)的受控生成任务上与最先进方法进行比较时,BOLT 展示了显著提高的效率和流畅性。在情感控制方面,BOLT 比有竞争力的基线快 7 倍,并且根据人类评估,在 74.4% 的评估样本中更加流畅。

英文摘要:

Energy-based models (EBMs) have gained popularity for controlled text generation due to their high applicability to a wide range of constraints. However, sampling from EBMs is non-trivial, as it often requires a large number of iterations to converge to plausible text, which slows down the decoding process and makes it less practical for real-world applications. In this work, we propose BOLT, which relies on tunable biases to directly adjust the language model's output logits. Unlike prior work, BOLT maintains the generator's autoregressive nature to assert a strong control on token-wise conditional dependencies and overall fluency, and thus converges faster. When compared with state-of-the-arts on controlled generation tasks using both soft constraints (e.g., sentiment control) and hard constraints (e.g., keyword-guided topic control), BOLT demonstrates significantly improved efficiency and fluency. On sentiment control, BOLT is 7x faster than competitive baselines, and more fluent in 74.4% of the evaluation samples according to human judges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑