了解大规模深度学习推荐模型的训练效率
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale
摘要：
GPU在机器学习工作流中的使用已经激增，现在被认为是许多深度学习模型的主流。与此同时，在训练最先进的个人推荐模型时，这些模型在我们的大规模计算中心消耗的计算周期最多，由于具有计算密集型和内存密集型组件，GPU的使用带来了各种挑战。这些推荐模型的GPU性能和效率在很大程度上受到模型架构配置的影响，如密集和稀疏特征，MLP维度。此外，这些模型通常包含不适合有限GPU内存的大型嵌入表。本文的目标是解释使用GPU训练推荐模型的复杂性，影响大规模硬件效率的因素，以及从新的扩展GPU服务器设计Zion中学习。
英文摘要：
The use of GPUs has proliferated for machine learning workflows and is now considered mainstream for many deep learning models. Meanwhile, when training state-of-the-art personal recommendation models, which consume the highest number of compute cycles at our large-scale datacenters, the use of GPUs came with various challenges due to having both compute-intensive and memory-intensive components. GPU performance and efficiency of these recommendation models are largely affected by model architecture configurations such as dense and sparse features, MLP dimensions. Furthermore, these models often contain large embedding tables that do not fit into limited GPU memory. The goal of this paper is to explain the intricacies of using GPUs for training recommendation models, factors affecting hardware efficiency at scale, and learnings from a new scale-up GPU server design, Zion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
