arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.12831cs.LG
收藏

DeepSpeed优化库的基准评估

Benchmark Assessment for DeepSpeed Optimization Library

Gongbo Liang, Izzat Alsmadi

摘要:

深度学习(DL)模型由于其性能和处理大型数据集的能力而广泛用于机器学习,同时产生高精度和性能指标。这样的数据集的大小和DL模型的复杂性导致这样的模型是复杂的,消耗大量的资源和时间来训练。许多最近的库和应用程序被引入来处理DL复杂性和效率问题。在本文中,我们评估了一个例子,微软DeepSpeed库通过分类任务。DeepSpeed公开源报告了LeNet架构上的分类性能指标。我们通过在几种现代神经网络架构上评估库来扩展这一点,包括卷积神经网络(CNN)和视觉Transformer(ViT)。结果表明,DeepSpeed虽然可以在某些情况下做出改进,但对其他情况没有影响或产生负面影响。

英文摘要:

Deep Learning (DL) models are widely used in machine learning due to their performance and ability to deal with large datasets while producing high accuracy and performance metrics. The size of such datasets and the complexity of DL models cause such models to be complex, consuming large amount of resources and time to train. Many recent libraries and applications are introduced to deal with DL complexity and efficiency issues. In this paper, we evaluated one example, Microsoft DeepSpeed library through classification tasks. DeepSpeed public sources reported classification performance metrics on the LeNet architecture. We extended this through evaluating the library on several modern neural network architectures, including convolutional neural networks (CNNs) and Vision Transformer (ViT). Results indicated that DeepSpeed, while can make improvements in some of those cases, it has no or negative impact on others.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑