Long-context capability has become essential for large language models (LLMs) to handle complex tasks, yet continuously expanding the context window does not remove the computational, storage, and information-selection pressures incurred at inference time. Existing work on prompt compression, KV-cache management, efficient serving, retrieval augmentation, and agent memory is usually discussed in isolation, leaving the flow of information between storage and computation insufficiently understood. This survey reframes long-context compression as a memory-management problem for allocating inference-time resources. We distinguish compute-active memory, which directly participates in model computation, from compute-inactive memory, which lies outside the computation path but can be stored, retrieved, updated, or forgotten, and we characterize offloaded KV caches, persistent prefix caches, and precomputed KV states as intermediate states on the activation path. On this basis, we formulate writing, offloading, recall, retrieval-and-injection, materialization, eviction, consolidation, and forgetting as state transitions within an information lifecycle, and introduce a budgeted cost model that places these operations in a single cost account. The resulting framework unifies short-term context compression, long-term memory management, and the transitions between them, and shows that the central challenge is not merely reducing information volume but deciding which information should participate in costly inference, when, and in what form. We further summarize evaluation dimensions, benchmark gaps, and open challenges, and argue that future research should move from static, pointwise compression toward dynamic, governable, lifecycle-aware memory management.
Shifting AI Efficiency From Model-Centric to Data-Centric Compression
EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Sichuan University(四川大学);University of Electronic Science & Technology of China(电子科技大学);Shanghai AI Laboratory(上海人工智能实验室);Sun Yat-sen University(中山大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddings
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);National University of Singapore(新加坡国立大学);Nanjing University of Posts and Telecommunications(南京邮电大学)
S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程研究中心)
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind
X-LANCE Lab, Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence, SJTU AI Institute Shanghai Jiao Tong University, Shanghai, China(X-LANCE实验室,计算机科学与工程系,MoE人工智能关键实验室,上海交通大学人工智能研究院,上海,中国);Suzhou Laboratory, Suzhou, China(苏州实验室,苏州,中国)
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University(信息处理国家重点实验室,计算机学院,PKU-Anker LLM实验室,北京大学);University of California, Los Angeles(加州大学洛杉矶分校);University of Washington(华盛顿大学);Georgia Institute of Technology(佐治亚理工学院);Nanyang Technological University(南洋理工大学);HKUST(香港科技大学);University of International Business and Economics(国际商务经济大学)
Ministry of Education Key Laboratory of Bioinformatics, BNRIST Bioinformatics Division, Department of Automation, Tsinghua University(教育部生物信息学重点实验室、BNRIST生物信息学分部、自动化系、清华大学);School of Computing and Artificial Intelligence, Fuyao University of Science and Technology(计算与人工智能学院、福耀科技大学);Department of Computer Science and Engineering, Southern University of Science and Technology(计算机科学与工程系、南方科技大学)
SEER: Long-Context Reasoning via Selective Visual-Text Compression
The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Cambridge(剑桥大学);University of Technology Sydney(悉尼科技大学);The University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校);The University of Texas at Austin(德克萨斯大学奥斯汀分校)