arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.00407cs.LGcs.PF
收藏

MIGPerf:多实例GPU上深度学习训练和推理工作负载的综合基准测试

MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs

Alibaba Group(阿里巴巴集团);Institute for Infocomm Research, A*STAR(新加坡科技研究局资讯通信研究院);Wuhan University(武汉大学);NVIDIA AI Tech Center(英伟达人工智能技术中心);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学)

Huaizheng Zhang, Yuanming Li, Wencong Xiao, Yizheng Huang, Xing Di, Jianxiong Yin, Simon See, Yong Luo, Chiew Tong Lau, Yang You

摘要:

像A100这样的新架构GPU现在配备了多实例GPU(GPU)技术,该技术允许将GPU划分为多个小型的独立实例。这项技术为用户提供了更大的灵活性,以支持深度学习训练和推理工作负载,但有效地利用它仍然具有挑战性。本文的目标是为XML提供一个更全面和实用的基准测试研究,以消除繁琐的手动基准测试和调优工作的需要。为了实现这一愿景,本文介绍了MIGPerf,一个开源工具,简化了基准测试研究。使用MIGPerf,作者进行了一系列实验,包括深度学习训练和推理表征,GPU共享表征,以及框架与GPU的兼容性。这些实验的结果为用户有效地使用多媒体数据库提供了新的见解和指导,并为进一步研究混合训练和推理工作负载在MIG上的编排奠定了基础。代码和结果发布在https://github.com/MLSysOps/MIGProfiler上。这项工作仍在进行中,不久将公布更多结果。

英文摘要:

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑