The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated models like OpenAI's ChatGPT, represents a significant advancement in artificial intelligence. These models, however, bring forth substantial challenges in the high consumption of computational, memory, energy, and financial resources, especially in environments with limited resource capabilities. This survey aims to systematically address these challenges by reviewing a broad spectrum of techniques designed to enhance the resource efficiency of LLMs. We categorize methods based on their optimization focus: computational, memory, energy, financial, and network resources and their applicability across various stages of an LLM's lifecycle, including architecture design, pretraining, finetuning, and system design. Additionally, the survey introduces a nuanced categorization of resource efficiency techniques by their specific resource types, which uncovers the intricate relationships and mappings between various resources and corresponding optimization techniques. A standardized set of evaluation metrics and datasets is also presented to facilitate consistent and fair comparisons across different models and techniques. By offering a comprehensive overview of the current sota and identifying open research avenues, this survey serves as a foundational reference for researchers and practitioners, aiding them in developing more sustainable and efficient LLMs in a rapidly evolving landscape.
Edge Intelligence for Energy-efficient Computation Offloading and Resource Allocation in 5G Beyond
University of Electronic Science and Technology of China(电子科技大学);University of Oslo(奥斯陆大学);Simula Metropolitan Center for Digital Engineering(Simula大都会数字工程中心)
Computation Offloading in Beyond 5G Networks: A Distributed Learning Framework and Applications
VTT Technical Research Centre of Finland(芬兰VTT技术研究中心);University of Electro-Communications(电气通信大学);Shizuoka University(静冈大学);University of Windsor(温莎大学);National Institute of Informatics(国立信息学研究所)
LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review
African Institute for Mathematical Sciences (AIMS)(非洲数学科学研究所);Bayero University Kano(卡诺贝罗大学);Brown University(布朗大学);Centrum Wiskunde & Informatica(数学与计算机科学中心);University of Pretoria(比勒陀利亚大学);University of Hamburg(汉堡大学);Imperial College London(伦敦帝国学院)
Adversarial Fragility and Language Vulnerability in Clinical AI: A Systematic Audit of Diagnostic Collapse Under Imperceptible Perturbations and Cross-Lingual Drift in Low-Resource Healthcare Settings
Centre for Clinical Intelligence & Safety(临床智能与安全中心);Tomorrow University of Applied Sciences(明天应用科学大学)