The efficiency of large language models (LLMs) remains a critical challenge, particularly in contexts where computational resources are limited. Traditional attention mechanisms in these models, while powerful, require significant computational and memory resources due to the necessity of recalculating and storing attention weights across different layers. This paper introduces a novel Shared Attention (SA) mechanism, designed to enhance the efficiency of LLMs by directly sharing computed attention weights across multiple layers. Unlike previous methods that focus on sharing intermediate Key-Value (KV) caches, our approach utilizes the isotropic tendencies of attention distributions observed in advanced LLMs post-pretraining to reduce both the computational flops and the size of the KV cache required during inference. We empirically demonstrate that implementing SA across various LLMs results in minimal accuracy loss on standard benchmarks. Our findings suggest that SA not only conserves computational resources but also maintains robust model performance, thereby facilitating the deployment of more efficient LLMs in resource-constrained environments.
KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
Technical University of Munich（慕尼黑技术大学）；Technical University of Darmstadt（达姆施塔特技术大学）；Zhejiang University（浙江大学）；Technische Universität Ilmenau（伊尔梅瑙技术大学）
LLM-Empowered Cooperative Content Caching in Vehicular Fog Caching-Assisted Platoon Networks
School of Internet of Things Engineering, Jiangnan University（江南大学物联网工程学院）；School of Information Engineering, Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications, Nanchang University（江西省级先进信号处理与智能通信重点实验室，南昌大学信息工程学院）；Department of Electronic Engineering, State Key Laboratory of Space Network and Communications, and the Beijing National Research Center for Information Science and Technology, Tsinghua University（电子工程系，空间网络与通信国家重点实验室，信息科学与技术国家研究中心，清华大学）；Department of Computer Science, Brunel University（计算机科学系，布鲁内尔大学）；State Key Laboratory of ISN and the School of Telecommunications Engineering, Xidian University（信息与通信国家重点实验室，西安电子科技大学电信工程学院）
d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（新一代人工智能技术及其交叉应用国家重点实验室（东南大学））；Southeast University（东南大学）；Qiyuan Tech（启元科技）
Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs
Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信（TeleAI）人工智能研究所）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）；Individual Researcher（独立研究者）
SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models
School of Computing Science, Simon Fraser University（西蒙菲莎大学计算科学学院）；University of Mississippi（密西西比大学）；Jiangxing Intelligence Inc.（江兴智联公司）；The University of Hong Kong（香港大学）
EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；School of Integrated Circuits, Peking University（北京大学集成电路学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Beijing Advanced Innovation Center for Integrated Circuits（北京集成电路先进制造创新中心）
WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering
Nanjing University of Science and Technology（南京理工大学）；Alibaba Group（阿里巴巴集团）；Huzhou Normal University（湖州师范学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs
Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信人工智能研究院（TeleAI））；Shanghai Jiao Tong University（上海交通大学）；University at Buffalo（纽约州立大学布法罗分校）
Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
Beijing Institute of Technology, China（北京理工大学）；Beijing University of Posts and Telecommunications, China（北京邮电大学）；Meituan, China（美团）；The Open University, UK（开放大学）
Cross-LLM Consistency in Inference: Evidence from Shared Interactions
School of Computer Science Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Ningbo Key Laboratory of Advanced Manufacturing Simulation Eastern Institute of Technology, Ningbo（宁波市先进制造仿真重点实验室，宁波东方理工大学）；College of Computer and Information Science Chongqing Normal University（重庆师范大学计算机与信息科学学院）；SymtrustAI.com；Eastern Institute of Technology, Ningbo（宁波东方理工大学）
