arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.09111cs.AIcs.LG
收藏

AI加速器上基础模型推理优化

Inference Optimization of Foundation Models on AI Accelerators

AWS AI(亚马逊云科技人工智能部门);EPFL(洛桑联邦理工学院)

Youngsuk Park, Kailash Budhathoki, Liangfu Chen, Jonas Kübler, Jiaji Huang, Matthäus Kleindessner, Jun Huan, Volkan Cevher, Yida Wang, George Karypis

AI总结:

本文是一篇教程,系统介绍基于AI加速器的大语言模型等基础模型推理优化技术,涵盖注意力计算优化、Transformer架构要素以及模型压缩与快速解码策略。

摘要:

强大的基础模型,包括采用Transformer架构的大语言模型(LLMs),已在各行各业开启了生成式AI的新时代。产业界和研究界见证了大量基于这些基础模型的新应用,包括问答、客户服务、图像与视频生成以及代码补全等。然而,随着模型参数量达到数千亿,其部署在现实场景中会产生高昂的推理成本和较高的延迟。因此,利用AI加速器实现高性价比且快速的推理需求日益迫切。为此,本教程对基于AI加速器的互补推理优化技术进行了全面讨论。首先概述基本的Transformer架构和深度学习系统框架,然后深入探讨用于快速且内存高效的注意力计算的系统优化技术,并讨论如何在AI加速器上高效实现这些技术。接下来,我们描述对快速Transformer推理至关重要的架构要素。最后,我们在同一背景下考察各种模型压缩和快速解码策略。

英文摘要:

Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries. Industry and research community have witnessed a large number of new applications, based on those foundation models. Such applications include question and answer, customer services, image and video generation, and code completions, among others. However, as the number of model parameters reaches to hundreds of billions, their deployment incurs prohibitive inference costs and high latency in real-world scenarios. As a result, the demand for cost-effective and fast inference using AI accelerators is ever more higher. To this end, our tutorial offers a comprehensive discussion on complementary inference optimization techniques using AI accelerators. Beginning with an overview of basic Transformer architectures and deep learning system frameworks, we deep dive into system optimization techniques for fast and memory-efficient attention computations and discuss how they can be implemented efficiently on AI accelerators. Next, we describe architectural elements that are key for fast transformer inference. Finally, we examine various model compression and fast decoding strategies in the same context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑