arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.04991cs.LGcs.CL
收藏

AIGC推理性能优化竞赛解决方案

The Solution for the AIGC Inference Performance Optimization Competition

Nanjing University of Science and Technology(南京理工大学)

Sishun Pan, Haonan Xu, Zhonghua Wan, Yang Yang

AI总结:

针对Ernie模型高性能推理需求,本文基于Paddle与GPU加速,结合Faster Transformer、embedding剪枝、FP16及多进程数据处理,实现最高8.96倍推理加速。

摘要:

近年来,基于Transformer架构的大规模预训练语言模型快速发展,为自然语言处理任务带来了革命性变化。其中,ChatGPT广受欢迎,展现出接近人类水平的对话能力,并在2022年底吸引了超过1亿月度用户。与此同时,百度对Ernie文心模型的商业化部署通过AI驱动技术显著提升了营销效果。本文聚焦于优化Ernie模型的高性能推理,重点关注GPU加速并利用Paddle推理框架。我们采用Faster Transformer等技术实现高效模型处理,通过embedding层剪枝降低计算开销,并使用FP16半精度推理提升计算效率。此外,我们的方法结合了使用多进程并行处理的高效数据处理策略,以最小化延迟。实验结果表明,与标准方法相比,我们的优化方案在保持有竞争力性能的同时,推理速度最高提升8.96倍。

英文摘要:

In recent years, the rapid advancement of large-scale pre-trained language models based on transformer architectures has revolutionized natural language processing tasks. Among these, ChatGPT has gained widespread popularity, demonstrating human-level conversational abilities and attracting over 100 million monthly users by late 2022. Concurrently, Baidu's commercial deployment of the Ernie Wenxin model has significantly enhanced marketing effectiveness through AI-driven technologies. This paper focuses on optimizing high-performance inference for Ernie models, emphasizing GPU acceleration and leveraging the Paddle inference framework. We employ techniques such as Faster Transformer for efficient model processing, embedding layer pruning to reduce computational overhead, and FP16 half-precision inference for enhanced computational efficiency. Additionally, our approach integrates efficient data handling strategies using multi-process parallel processing to minimize latency. Experimental results demonstrate that our optimized solution achieves up to an 8.96x improvement in inference speed compared to standard methods, while maintaining competitive performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑