arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.00855cs.CLstat.ML
收藏

LanguaShrink:利用心理语言学减少令牌开销

LanguaShrink: Reducing Token Overhead with Psycholinguistics

East China Jiaotong University(华东交通大学);Guangdong University of Technology(广东工业大学);University of Toronto(多伦多大学);East China Normal University(华东师范大学)

Xuechen Liang, Meiling Tao, Yinghui Xia, Tianyu Shi, Jun Wang, JingSong Yang

AI总结:

LanguaShrink框架利用心理语言学原理和艾宾浩斯记忆曲线,结合词性优先级压缩、数据蒸馏及KL正则化强化学习,实现任务无关的提示词压缩,在保持语义的同时达到最高26倍压缩,并将端到端延迟提升1.43倍。

摘要:

随着大语言模型(LLMs)处理复杂任务的能力不断提升,长提示词带来的计算成本与效率问题日益突出。为了加速模型推理并降低成本,我们提出了一种创新的提示词压缩框架,名为LanguaShrink。受LLM性能取决于输入提示中关键信息的密度和位置这一观察启发,LanguaShrink利用心理语言学原理和艾宾浩斯记忆曲线实现任务无关的提示词压缩,在保留关键信息的同时有效缩短提示长度。我们参考了OpenChat的训练方法。该框架引入词性优先级压缩和数据蒸馏技术,使用较小的模型学习压缩目标,并采用KL正则化强化学习策略进行训练。此外,我们采用基于块的压缩算法以实现可调节的压缩率。我们在多个数据集上评估了该方法,包括LongBench、ZeroScrolls、Arxiv Articles以及一个新构建的小说测试集。实验结果表明,LanguaShrink在保持语义相似性的同时实现了高达26倍的压缩。与现有提示词压缩方法相比,LanguaShrink将端到端延迟提升了1.43倍。

英文摘要:

As large language models (LLMs) improve their capabilities in handling complex tasks, the issues of computational cost and efficiency due to long prompts are becoming increasingly prominent. To accelerate model inference and reduce costs, we propose an innovative prompt compression framework called LanguaShrink. Inspired by the observation that LLM performance depends on the density and position of key information in the input prompts, LanguaShrink leverages psycholinguistic principles and the Ebbinghaus memory curve to achieve task-agnostic prompt compression. This effectively reduces prompt length while preserving essential information. We referred to the training method of OpenChat.The framework introduces part-of-speech priority compression and data distillation techniques, using smaller models to learn compression targets and employing a KL-regularized reinforcement learning strategy for training.\cite{wang2023openchat} Additionally, we adopt a chunk-based compression algorithm to achieve adjustable compression rates. We evaluate our method on multiple datasets, including LongBench, ZeroScrolls, Arxiv Articles, and a newly constructed novel test set. Experimental results show that LanguaShrink maintains semantic similarity while achieving up to 26 times compression. Compared to existing prompt compression methods, LanguaShrink improves end-to-end latency by 1.43 times.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑