Compressed prompts aid instruction-tuned language models (LMs) in overcoming context window limitations and reducing computational costs. Existing methods, which primarily based on training embeddings, face various challenges associated with interpretability, the fixed number of embedding tokens, reusability across different LMs, and inapplicability when interacting with black-box APIs. This study proposes prompt compression with reinforcement learning (PCRL), which is a discrete prompt compression method that addresses these issues. The proposed PCRL method utilizes a computationally efficient policy network that edits prompts directly. The training approach employed in the proposed PCRLs can be applied flexibly to various types of LMs, including both decoder-only and encoder-decoder architecture and it can be trained without gradient access to the LMs or labeled data. The proposed PCRL achieves an average reduction of 24.6% in terms of the token count across various instruction prompts while maintaining sufficient performance. In addition, we demonstrate that the learned policy can be transferred to larger LMs, and through a comprehensive analysis, we explore the token importance within the prompts. Our code is accessible at https://github.com/nenomigami/PromptCompressor.
Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization
CFCS, School of Computer Science, Peking University（计算机学院，北京大学）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；LLM-Core Xiaomi（小米LLM-Core）
QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning
Tsinghua University（清华大学）；SenseTime Research（商汤科技研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Sun Yat-Sen University（中山大学）；Nanyang Technological University（南洋理工大学）
Sequence Compression Speeds Up Credit Assignment in Reinforcement Learning
The Swiss AI Lab IDSIA, USI & SUPSI（瑞士IDSIA人工智能实验室、瑞士意大利语区大学、瑞士南部应用科学大学）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；AI Initiative, King Abdullah University of Science and Technology（阿卜杜拉国王科技大学人工智能计划）
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Department of Statistics（统计学系）；Data Science, Northwestern University（数据科学，西北大学）；Bridgewater AIA Labs（布里奇沃特AIA实验室）
Green Deep Reinforcement Learning for Radio Resource Management: Architecture, Algorithm Compression and Challenge
National University of Defense Technology（中国人民解放军国防科技大学）；Kings College London（伦敦国王学院）；Cranfield University（克兰菲尔德大学）；Alan Turing Institute（艾伦·图灵研究所）；Queen Mary University of London（伦敦玛丽女王大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Contextual Transformer for Offline Meta Reinforcement Learning
University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；University College London（伦敦大学学院）；Chinese Academy of Sciences（中国科学院）；King’s College London（伦敦国王学院）
An Empirical Study on Prompt Compression for Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；South China University of Technology（华南理工大学）；University of Science and Technology of China（中国科学技术大学）
PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Zhejiang University（浙江大学）；CAS Key Laboratory of AI Security, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）；Fudan University（复旦大学）
Ministry of Education Key Laboratory of Bioinformatics, BNRIST Bioinformatics Division, Department of Automation, Tsinghua University（教育部生物信息学重点实验室、BNRIST生物信息学分部、自动化系、清华大学）；School of Computing and Artificial Intelligence, Fuyao University of Science and Technology（计算与人工智能学院、福耀科技大学）；Department of Computer Science and Engineering, Southern University of Science and Technology（计算机科学与工程系、南方科技大学）
Dynamic Compressing Prompts for Efficient Inference of Large Language Models
School of Software Engineering, South China University of Technology（华南理工大学软件学院）；Pazhou Lab（琶洲实验室）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；Peng Cheng Laboratory（鹏城实验室）；Hong Kong Polytechnic University（香港理工大学）；Department of Computer Science and Technology, Chongqing University of Posts and Telecommunications（重庆邮电大学计算机科学与技术学院）
Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明的负责任人工智能与数据分析（PRADA）实验室）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；South China University of Technology（华南理工大学）；University of Macau（澳门大学）；Nankai University（南开大学）；University of Copenhagen（哥本哈根大学）；Stony Brook University（纽约州立大学石溪分校）；SDAIA-KAUST AI（沙特数据与人工智能总局-阿卜杜拉国王科技大学人工智能机构）；Ant Group（蚂蚁集团）
