PositionID:大语言模型可通过显式位置感知实现长度控制与复制粘贴
PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
AI总结:
针对大语言模型长度控制能力不足的问题,该研究提出PositionID系列提示与微调方法增强位置感知,还构建相关基准,实验证明其可在不降低响应质量的前提下提升长度控制和复制粘贴准确率。
摘要:
大语言模型(LLMs)在角色扮演、创意写作、数学推理和编程等多个领域展现出出色能力。尽管取得了这些进展,LLMs在长度控制方面仍面临挑战,由于其基于token的运算机制,且在严格长度限制的数据上训练不足,经常无法遵守特定的长度约束。我们发现该问题源于位置感知的缺失,为此提出了PositionID提示(PositionID Prompting)和PositionID微调(PositionID Fine-Tuning)两种新方法来解决这一问题,这些方法增强了模型在生成过程中持续监控和管理文本长度的能力。此外,我们提出PositionID CP提示(PositionID CP Prompting),使LLMs能够准确执行复制粘贴操作。我们还构建了两个基准,用于评估长度控制和复制粘贴能力。实验表明,我们的方法在不降低响应质量的前提下,显著提升了模型对长度约束的遵守程度和复制粘贴准确率。
英文摘要:
Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches--PositionID Prompting and PositionID Fine-Tuning--to address it. These methods enhance the model's ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model's adherence to length constraints and copy-paste accuracy without compromising response quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。