arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.01990cs.DCcs.LG
收藏

高效训练与推理的大型基础模型设计:综述

Designing Large Foundation Models for Efficient Training and Inference: A Survey

University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Columbia University(哥伦比亚大学);University of Illinois at Urbana–Champaign(伊利诺伊大学厄巴纳-香槟分校);The Ohio State University(俄亥俄州立大学);Iran University of Science and Technology(伊朗科技大学)

Dong Liu, Yanxuan Yu, Yite Wang, Jing Wu, Zhongwei Wan, Sina Alinejad, Benjamin Lengerich, Ying Nian Wu

AI总结:

本文综述基础模型的高效训练与推理技术,从模型和系统设计双视角探讨优化方法以节省计算资源,并提供相关资源库,助力LLM更高效、经济且易获取。

摘要:

本文聚焦基础模型的现代高效训练与推理技术,从模型设计和系统设计两个视角展开阐述。二者从不同维度优化大语言模型(LLM)的训练与推理,以节省计算资源,使LLM更高效、经济且易于获取。论文相关资源库可通过https://github.com/NoakLiu/Efficient-Foundation-Models-Survey访问。

英文摘要:

This paper focuses on modern efficient training and inference technologies on foundation models and illustrates them from two perspectives: model and system design. Model and System Design optimize LLM training and inference from different aspects to save computational resources, making LLMs more efficient, affordable, and more accessible. The paper list repository is available at https://github.com/NoakLiu/Efficient-Foundation-Models-Survey.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑