A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Fudan University（复旦大学）；East China Normal University（华东师范大学）；Ping An Health and Technology Company Limited（平安健康科技有限公司）；Ping An Healthcare Technology（平安医疗科技）；Ping An International Smart City Technology Company Limited（平安国际智慧城市科技有限公司）；Peng Cheng Laboratory（鹏城实验室）
Early exiting allows instances to exit at different layers according to the estimation of difficulty. Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning. In contrast, learning to exit, or learning to predict instance difficulty is a more appealing way. Though some effort has been devoted to employing such "learn-to-exit" modules, it is still unknown whether and how well the instance difficulty can be learned. As a response, we first conduct experiments on the learnability of instance difficulty, which demonstrates that modern neural models perform poorly on predicting instance difficulty. Based on this observation, we propose a simple-yet-effective Hash-based Early Exiting approach (HashEE) that replaces the learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. Different from previous methods, HashEE requires no internal classifiers nor extra parameters, and therefore is more efficient. Experimental results on classification, regression, and generation tasks demonstrate that HashEE can achieve higher performance with fewer FLOPs and inference time compared with previous state-of-the-art early exiting methods.
DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks
School of Information Engineering, Tianjin University of Commerce（天津商业大学信息工程学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；DataX Research Centre, School of Computing, Macquarie University（麦考瑞大学计算学院DataX研究中心）
You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model
North Carolina State University（北卡罗来纳州立大学）；Google Research（谷歌研究院）；Northeastern University（东北大学）；University of Michigan（密歇根大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Connecticut（康涅狄格大学）
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；MOE Key Laboratory of Computational Linguistics, Peking University（北京大学计算语言学教育部重点实验室）
Training dynamic models using early exits for automatic speech recognition on resource-constrained devices
University of Trento（特伦托大学）；LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎综合理工学院电信巴黎分校LTCI实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）；Universite Paris-Saclay, LISN, CNRS（巴黎萨克雷大学LISN实验室，法国国家科学研究中心）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
Two-dimensional early exit optimisation of LLM inference
Institute for Research and Applications of Fuzzy Modelling（模糊建模研究与应用研究所）；University of Ostrava（奥斯特拉瓦大学）；Institute of Computer Science（计算机科学研究所）；Faculty of Philosophy and Science（哲学与科学学院）；Silesian University in Opava（奥帕瓦西里西亚大学）
Dynamic Transformers Provide a False Sense of Efficiency
National University of Singapore（新加坡国立大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Kriston AI Lab（克里斯顿人工智能实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of California, Riverside（加州大学河滨分校）
What Affects the Effective Depth of Large Language Models?
Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）；Department of Computer Science, Massachusetts Institute of Technology（计算机科学系，麻省理工学院）
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
Amirkabir University of Technology（阿米尔卡比尔理工大学）；Stanford University（斯坦福大学）；Massachusetts Institute of Technology（麻省理工学院）；Columbia University（哥伦比亚大学）；Amazon GenAI（亚马逊生成式AI部门）
