Neural language models (NLMs) exist in an accuracy-efficiency tradeoff space where better perplexity typically comes at the cost of greater computation complexity. In a software keyboard application on mobile devices, this translates into higher power consumption and shorter battery life. This paper represents the first attempt, to our knowledge, in exploring accuracy-efficiency tradeoffs for NLMs. Building on quasi-recurrent neural networks (QRNNs), we apply pruning techniques to provide a "knob" to select different operating points. In addition, we propose a simple technique to recover some perplexity using a negligible amount of memory. Our empirical evaluations consider both perplexity as well as energy consumption on a Raspberry Pi, where we demonstrate which methods provide the best perplexity-power consumption operating point. At one operating point, one of the techniques is able to provide energy savings of 40% over the state of the art with only a 17% relative increase in perplexity.
Fluctuation-based Adaptive Structured Pruning for Large Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Wuhan AI Research（武汉人工智能研究院）；Objecteye Inc.（Objecteye公司）
Adaptive Pruning for Large Language Models with Structural Importance Awareness
Harbin Engineering University（哈尔滨工程大学）；Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)（深圳未来网络智能研究院（香港中文大学（深圳）））；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）
Adaptive Graph Pruning for Multi-Agent Communication
Zhejiang University - University of Illinois Urbana-Champaign Institute（浙江大学 - 印第安纳大学厄巴纳-香槟分校研究所）；Zhejiang University, College of Computer Science and Technology（浙江大学计算机科学与技术学院）
