Prompt compression is crucial for enhancing inference speed, reducing costs, and improving user experience. However, current methods face challenges such as low compression ratios and potential data leakage during evaluation. To address these issues, we propose 500xCompressor, a method that compresses extensive natural language contexts into a minimum of one single special token. The 500xCompressor introduces approximately 0.3% additional parameters and achieves compression ratios ranging from 6x to 480x. It is designed to compress any text, answer various types of questions, and could be utilized by the original large language model (LLM) without requiring fine-tuning. Initially, 500xCompressor was pretrained on the Arxiv Corpus, followed by fine-tuning on the ArxivQA dataset, and subsequently evaluated on strictly unseen and classical question answering (QA) datasets. The results demonstrate that the LLM retained 62.26-72.89% of its capabilities compared to using non-compressed prompts. This study also shows that not all the compressed tokens are equally utilized and that K V values have significant advantages over embeddings in preserving information at high compression ratios. The highly compressive nature of natural language prompts, even for fine-grained complex information, suggests promising potential for future applications and further research into developing a new LLM language.
DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University（上海教育委员会智能交互与认知工程重点实验室）；Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3（上海Web3可信数据流通与治理重点实验室）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Xiaomi, Beijing, China（小米（北京））
PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）
Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；Ant Group（蚂蚁集团）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）
An Empirical Study on Prompt Compression for Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；University of Science and Technology of China（中国科学技术大学）
Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Zhejiang University（浙江大学）；CAS Key Laboratory of AI Security, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）；Fudan University（复旦大学）
Ministry of Education Key Laboratory of Bioinformatics, BNRIST Bioinformatics Division, Department of Automation, Tsinghua University（教育部生物信息学重点实验室、BNRIST生物信息学分部、自动化系、清华大学）；School of Computing and Artificial Intelligence, Fuyao University of Science and Technology（计算与人工智能学院、福耀科技大学）；Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系、南方科技大学）
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
School of Computer Technology and Application, Qinghai University（青海大学计算机技术与应用学院）；Tsinghua University（清华大学）；Ant Group Security and Intelligence Laboratory (SIL)（蚂蚁集团安全与智能实验室）
Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National University of Singapore（新加坡国立大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明的负责任人工智能与数据分析（PRADA）实验室）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；South China University of Technology（华南理工大学）；University of Macau（澳门大学）；Nankai University（南开大学）；University of Copenhagen（哥本哈根大学）；Stony Brook University（纽约州立大学石溪分校）；SDAIA-KAUST AI（沙特数据与人工智能总局-阿卜杜拉国王科技大学人工智能机构）；Ant Group（蚂蚁集团）
Dynamic Compressing Prompts for Efficient Inference of Large Language Models
School of Software Engineering, South China University of Technology（华南理工大学软件学院）；Pazhou Lab（琶洲实验室）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；Peng Cheng Laboratory（鹏城实验室）；Hong Kong Polytechnic University（香港理工大学）；Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications（重庆邮电大学计算机科学与技术学院）
DSPC: Dual-Stage Progressive Compression Framework for Efficient Long-Context Reasoning
Institute of Cyberspace Security, Zhejiang University of Technology A STAR Amazon Binjiang Institute of Artificial Intelligence, Zhejiang University of Technology
S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Engineering Research Center of Intelligent Vision and Imaging（上海智能视觉与成像工程研究中心）
