arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.14033cs.CLcs.LG
收藏

Hansel:面向大型语言模型的输出长度控制框架

Hansel: Output Length Controlling Framework for Large Language Models

Samsung Research(三星研究院)

Seoha Song, Junhyun Lee, Hyeonmok Ko

AI总结:

本文提出Hansel框架,通过周期性输出隐藏特殊标记跟踪剩余目标长度,在不影响生成能力的前提下实现LLMs的输出长度控制,显著降低误差并提升对未见目标长度的泛化能力。

摘要:

尽管大型语言模型(LLMs)取得了巨大成功,但高效控制输出序列的长度仍然是一个挑战。在本文中,我们提出了Hansel,一个用于LLMs长度控制的高效框架,且不影响其生成能力。Hansel利用周期性输出的隐藏特殊标记来跟踪输出序列的剩余目标长度。结合避免输出突然终止的技术,这种看似简单的方法被证明是高效且通用的,同时不损害生成文本的连贯性和流畅性。该框架可在模型的微调阶段应用于任何预训练LLMs,无论其原始位置编码方法如何。我们通过对四个不同的LLMs进行Hansel微调来证明这一点,并表明与基于提示的长度控制微调相比,每个模型和数据集的输出序列平均绝对误差均显著降低。此外,该框架在推断微调期间未见的目标长度(如长对话回复或极短摘要)方面表现出显著提升的能力。这表明模型学习到了长度控制的通用方法,而不是学习将输出长度与训练期间见到的长度相匹配。

英文摘要:

Despite the great success of large language models (LLMs), efficiently controlling the length of the output sequence still remains a challenge. In this paper, we propose Hansel, an efficient framework for length control in LLMs without affecting its generation ability. Hansel utilizes periodically outputted hidden special tokens to keep track of the remaining target length of the output sequence. Together with techniques to avoid abrupt termination of the output, this seemingly simple method proved to be efficient and versatile, while not harming the coherency and fluency of the generated text. The framework can be applied to any pre-trained LLMs during the finetuning stage of the model, regardless of its original positional encoding method. We demonstrate this by finetuning four different LLMs with Hansel and show that the mean absolute error of the output sequence decreases significantly in every model and dataset compared to the prompt-based length control finetuning. Moreover, the framework showed a substantially improved ability to extrapolate to target lengths unseen during finetuning, such as long dialog responses or extremely short summaries. This indicates that the model learns the general means of length control, rather than learning to match output lengths to those seen during training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑