arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.02508cs.LGcs.AIcs.DC
收藏

追求低碳电力以实现实用且可持续的DNN训练

Chasing Low-Carbon Electricity for Practical and Sustainable DNN Training

University of Michigan(密歇根大学)

Zhenning Yang, Luoxi Meng, Jae-Won Chung, Mosharaf Chowdhury

AI总结:

提出Chase解决方案,通过实时监测碳强度并控制GPU能耗,在不迁移或推迟任务的情况下减少DNN训练碳足迹,同时利用轻量级ML算法预测碳强度,在ImageNet上训练ResNet-50时减少13.6%碳排放且仅增加2.5%训练时间。

摘要:

近年来,深度学习经历了显著增长,导致因使用GPU训练深度神经网络(DNNs)而产生的能耗和碳排放增加。响应可持续性的呼声,传统解决方案尝试将训练任务迁移到具有较低碳强度的地点或时间段。然而,由于数据集规模庞大或数据法规限制,将任务迁移到其他地点可能并不总是可行的。此外,推迟训练会对应用服务质量产生负面影响,因为支撑该服务的DNNs无法及时更新。在本研究中,我们提出了一种实用解决方案,能够在不迁移或推迟任务的情况下减少DNN训练的碳足迹。具体而言,我们的解决方案在训练过程中观察实时碳强度变化并控制GPU的能耗,从而在保持训练性能的同时减少碳足迹。此外,为了主动适应碳强度的变化,我们提出了一种轻量级机器学习算法,用于预测即将到来时间段的碳强度。我们的解决方案Chase,在ImageNet上训练ResNet-50时,将总碳足迹减少了13.6%,同时仅增加了2.5%的训练时间。

英文摘要:

Deep learning has experienced significant growth in recent years, resulting in increased energy consumption and carbon emission from the use of GPUs for training deep neural networks (DNNs). Answering the call for sustainability, conventional solutions have attempted to move training jobs to locations or time frames with lower carbon intensity. However, moving jobs to other locations may not always be feasible due to large dataset sizes or data regulations. Moreover, postponing training can negatively impact application service quality because the DNNs backing the service are not updated in a timely fashion. In this work, we present a practical solution that reduces the carbon footprint of DNN training without migrating or postponing jobs. Specifically, our solution observes real-time carbon intensity shifts during training and controls the energy consumption of GPUs, thereby reducing carbon footprint while maintaining training performance. Furthermore, in order to proactively adapt to shifting carbon intensity, we propose a lightweight machine learning algorithm that predicts the carbon intensity of the upcoming time frame. Our solution, Chase, reduces the total carbon footprint of training ResNet-50 on ImageNet by 13.6% while only increasing training time by 2.5%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑