arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.23271cs.CL
收藏

行为等效标记:用于LLMs长提示的单标记替代

Learning a Single Token to Replace Long System Prompts in LLMs

City University of Hong Kong(香港城市大学);Baidu Inc.(百度公司)

Jiancheng Dong, Pengyue Jia, Jingyu Peng, Maolin Wang, Yuhao Wang, Lixin Su, Xin Sun, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

AI总结:

本文提出了一种轻量级框架,通过学习单个行为等效标记替代长提示,实现提示长度减少3000倍并保持98%的下游性能。

摘要:

精心设计的系统提示在指导LLM代理行为方面起着关键作用,但其长度较长会带来显著的弊端,包括增加推理延迟、更高的计算成本和减少的有效上下文长度。这引发了这样一个问题:是否可以将这些长提示用大幅减少的标记数来替代,同时保持其对下游任务的行为效果?为此,我们提出了一种轻量级的三阶段训练框架,学习一个特定于提示的Behavior-Equivalent token([BE])。该框架首先训练[BE]通过重建来编码原始系统提示的自然语言内容,然后将提示的下游行为蒸馏到这个单个标记中。重要的是,我们的方法不需要访问模型内部、不需要辅助压缩模型、也不需要标记响应。在三个数据集上的实证评估表明,一个[BE]标记可以将提示长度减少多达3000倍,同时保留原始系统提示约98%的下游性能。这大大减少了推理成本,并几乎让整个上下文窗口可用于用户输入。

英文摘要:

Long system prompts are widely used to steer Large Language Models (LLMs), but repeatedly processing them at inference time is inefficient and consumes valuable context budget. This motivates a central question: can the behavioral effect of a long system prompt be retained using only a minimal learned representation? To enable this, we propose a lightweight training framework that learns a single Behavior-Equivalent Token ([BE]). The framework first trains [BE] to encode the semantic content of the original system prompt via reconstruction, and then distills the prompt's downstream behavior into this single token. Importantly, our method requires no update to the pretrained LLM weights, no auxiliary compression models, and no labeled responses. Empirical evaluations on three datasets show that replacing long prompts with a single [BE] token yields up to a $3000\times$ prompt compression ratio, while retaining about 98% of the downstream performance of the original system prompts. This substantially reduces inference cost and frees nearly the entire context window for user inputs and model outputs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑