Long context inference presents challenges at the system level with increased compute and memory requirements, as well as from an accuracy perspective in being able to reason over long contexts. Recently, several methods have been proposed to compress the prompt to reduce the context length. However, there has been little work on comparing the different proposed methods across different tasks through a standardized analysis. This has led to conflicting results. To address this, here we perform a comprehensive characterization and evaluation of different prompt compression methods. In particular, we analyze extractive compression, summarization-based abstractive compression, and token pruning methods. Surprisingly, we find that extractive compression often outperforms all the other approaches, and enables up to 10x compression with minimal accuracy degradation. Interestingly, we also find that despite several recent claims, token pruning methods often lag behind extractive compression. We only found marginal improvements on summarization tasks.
Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明的负责任人工智能与数据分析（PRADA）实验室）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；South China University of Technology（华南理工大学）；University of Macau（澳门大学）；Nankai University（南开大学）；University of Copenhagen（哥本哈根大学）；Stony Brook University（纽约州立大学石溪分校）；SDAIA-KAUST AI（沙特数据与人工智能总局-阿卜杜拉国王科技大学人工智能机构）；Ant Group（蚂蚁集团）
DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University（上海教育委员会智能交互与认知工程重点实验室）；Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3（上海Web3可信数据流通与治理重点实验室）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Xiaomi, Beijing, China（小米（北京））
An Empirical Study on Prompt Compression for Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；University of Science and Technology of China（中国科学技术大学）
PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）
PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Zhejiang University（浙江大学）；CAS Key Laboratory of AI Security, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）；Fudan University（复旦大学）
Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；Ant Group（蚂蚁集团）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）
Ministry of Education Key Laboratory of Bioinformatics, BNRIST Bioinformatics Division, Department of Automation, Tsinghua University（教育部生物信息学重点实验室、BNRIST生物信息学分部、自动化系、清华大学）；School of Computing and Artificial Intelligence, Fuyao University of Science and Technology（计算与人工智能学院、福耀科技大学）；Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系、南方科技大学）
Dynamic Compressing Prompts for Efficient Inference of Large Language Models
School of Software Engineering, South China University of Technology（华南理工大学软件学院）；Pazhou Lab（琶洲实验室）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；Peng Cheng Laboratory（鹏城实验室）；Hong Kong Polytechnic University（香港理工大学）；Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications（重庆邮电大学计算机科学与技术学院）
Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
