学习使用Gist Token压缩提示词
Learning to Compress Prompts with Gist Tokens
AI总结：
提出gisting方法，通过训练语言模型将提示词压缩为可缓存的gist token，实现高达26倍的压缩比，显著降低计算开销并且保持输出质量。
摘要：
提示是利用语言模型（LMs）多任务能力的主要方式，但提示词占用输入上下文窗口中的宝贵空间，且重复编码相同提示词在计算上效率低下。微调和与蒸馏方法允许在不使用提示词的情况下对LMs进行特化，但需为每个任务重新训练模型。为完全避免这种权衡，我们提出了gisting方法，该方法训练LM将提示词压缩为更小的“gist” token集合，这些token可被缓存并重复使用以提高计算效率。Gist模型的训练成本与标准指令微调无额外开销，仅需修改Transformer注意力掩码以鼓励提示词压缩。在解码器（LLaMA-7B）和编码器-解码器（FLAN-T5-XXL）LMs上，gisting实现了高达26倍的提示词压缩，带来高达40%的FLOPs减少、4.2%的实际运行时间加速以及存储节省，且输出质量损失失极小。
英文摘要：
Prompting is the primary way to utilize the multitask capabilities of language models (LMs), but prompts occupy valuable space in the input context window, and repeatedly encoding the same prompt is computationally inefficient. Finetuning and distillation methods allow for specialization of LMs without prompting, but require retraining the model for each task. To avoid this trade-off entirely, we present gisting, which trains an LM to compress prompts into smaller sets of "gist" tokens which can be cached and reused for compute efficiency. Gist models can be trained with no additional cost over standard instruction finetuning by simply modifying Transformer attention masks to encourage prompt compression. On decoder (LLaMA-7B) and encoder-decoder (FLAN-T5-XXL) LMs, gisting enables up to 26x compression of prompts, resulting in up to 40% FLOPs reductions, 4.2% wall time speedups, and storage savings, all with minimal loss in output quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
