The efficient implementation of large language models (LLMs) is crucial for deployment on resource-constrained devices. Low-rank tensor compression techniques, such as tensor-train (TT) networks, have been widely studied for over-parameterized neural networks. However, their applications to compress pre-trained large language models (LLMs) for downstream tasks (post-training) remains challenging due to the high-rank nature of pre-trained LLMs and the lack of access to pretraining data. In this study, we investigate low-rank tensorized LLMs during fine-tuning and propose sparse augmented tensor networks (Saten) to enhance their performance. The proposed Saten framework enables full model compression. Experimental results demonstrate that Saten enhances both accuracy and compression efficiency in tensorized language models, achieving state-of-the-art performance.
TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction
Meetyou AI Lab（美柚人工智能实验室）；Xiamen Key Laboratory of Women’s Internet Health Management（厦门市女性互联网健康管理重点实验室）；Osaka University（大阪大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）
AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算先进创新中心）；School of Artificial Intelligence, Beihang University（北航人工智能学院）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
Centre for Responsible AI（负责任人工智能中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Department of Electrical Engineering（电气工程系）；Wadhwani School of Data Science and Artificial Intelligence（Wadhwani数据科学与人工智能学院）
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National University of Singapore（新加坡国立大学）
Concept than Document: Context Compression via AMR-based Conceptual Entropy
University of Southern Queensland（南方昆士兰大学）；University of Technology Sydney（技术悉尼大学）；The Hong Kong Polytechnic University（香港理工大学）；Wuhan University of Technology（武汉理工大学）；National University of Singapore（新加坡国立大学）；The Education University of Hong Kong（香港教育大学）
AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation
Key Lab of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (CAS)（智能信息处理重点实验室，计算技术研究所，中国科学院）；State Key Lab of AI Safety（人工智能安全国家重点实验室）；University of Chinese Academy of Sciences, CAS（中国科学院大学）
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Baidu Inc.（百度公司）；Tencent Inc.（腾讯公司）
Rethinking Soft Compression in Retrieval-Augmented Generation: A Query-Conditioned Selector Perspective
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence（上海数据科学 key 实验室，计算机科学与人工智能学院）；Fudan University（复旦大学）；Shanghai（上海）；China（中国）
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
Sun Yat-sen University, China（中山大学）；Huawei Cloud Computing Technologies Co., Ltd., China（华为云计算技术有限公司）；Chongqing University, China（重庆大学）；University of Electronic Science and Technology of China, China（电子科技大学）
Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
Meta；Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（Inria 巴黎-罗克quencourt 分所）；Rajiv Gandhi University（拉朱·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司人工智能实验室）；University College London（伦敦大学学院）
Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
Ping An Technology (Shenzhen) Co., Ltd., China（平安科技（深圳）有限公司，中国）；University of Science and Technology of China（中国科学技术大学）；University of Electronic Science and Technology of China（电子科技大学）
Lighter And Better: Towards Flexible Context Adaptation For Retrieval Augmented Generation
University of Science and Technology of China（中国科学技术大学）；Gaoling School of AI, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
