arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18700cs.CLcs.AIcs.LG
收藏

学习以自然语言格式压缩提示词

Learning to Compress Prompt in Natural Language Formats

Rice University(莱斯大学);Samsung Research America(三星美国研究院);Texas A&M University(德克萨斯农工大学)

Yu-Neng Chuang, Tianwei Xing, Chia-Yuan Chang, Zirui Liu, Xun Chen, Xia Hu

AI总结:

本文提出Nano-Capsulator框架,将长提示压缩为自然语言格式的Capsule Prompt,通过语义保持损失和长度约束奖励函数优化,在保持可迁移性的同时减少81.4%长度并显著降低推理延迟与成本。

摘要:

大型语言模型(LLMs)擅长处理多种自然语言处理任务,但其能力受到长上下文性能较差、推理速度慢以及计算结果成本高昂的制约。以精确且信息丰富的上下文部署LLMs有助于用户更有效且更具成本效益地处理大规模数据集。现有工作依赖于将长提示上下文压缩为软提示(soft prompts)。然而,软提示压缩在不同LLMs之间的可迁移性方面存在局限,尤其是基于API的LLMs。为此,本工作旨在以自然语言形式压缩冗长提示,并具备LLM可迁移性。这带来两个挑战:(i) 自然语言(NL)提示与反向传播不兼容;(ii) NL提示在施加长度约束方面缺乏灵活性。在本工作中,我们提出了一种自然语言提示封装(Nano-Capsulator)框架,将原始提示压缩为NL格式的Capsule Prompt,同时保持提示的效用和可迁移性。具体而言,为解决第一个挑战,Nano-Capsulator通过一个与所提出的语义保持损失交互的奖励函数进行优化。为解决第二个问题,Nano-Capsulator通过一个具有长度约束特征的奖励函数进行优化。实验结果表明,Capsule Prompt能够减少81.4%的原始长度,将推理延迟降低至多4.5倍,并节省80.1%的预算开销,同时在不同LLMs和不同数据集之间提供可迁移性。

英文摘要:

Large language models (LLMs) are great at processing multiple natural language processing tasks, but their abilities are constrained by inferior performance with long context, slow inference speed, and the high cost of computing the results. Deploying LLMs with precise and informative context helps users process large-scale datasets more effectively and cost-efficiently. Existing works rely on compressing long prompt contexts into soft prompts. However, soft prompt compression encounters limitations in transferability across different LLMs, especially API-based LLMs. To this end, this work aims to compress lengthy prompts in the form of natural language with LLM transferability. This poses two challenges: (i) Natural Language (NL) prompts are incompatible with back-propagation, and (ii) NL prompts lack flexibility in imposing length constraints. In this work, we propose a Natural Language Prompt Encapsulation (Nano-Capsulator) framework compressing original prompts into NL formatted Capsule Prompt while maintaining the prompt utility and transferability. Specifically, to tackle the first challenge, the Nano-Capsulator is optimized by a reward function that interacts with the proposed semantics preserving loss. To address the second question, the Nano-Capsulator is optimized by a reward function featuring length constraints. Experimental results demonstrate that the Capsule Prompt can reduce 81.4% of the original length, decrease inference latency up to 4.5x, and save 80.1% of budget overheads while providing transferability across diverse LLMs and different datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑