arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.17752cs.DCcs.PF
收藏

多实例间 GPU 功耗分配研究

On the Partitioning of GPU Power among Multi-Instances

Tirth Vamja, Kaustabha Ray, Felix George, UmaMaheswari C Devi

AI总结:

针对 MIG 缺少硬件级功耗分摊支持的问题,本文提出基于分区利用率指标的在线 ML 功耗模型,并在 A100 上验证其可准确估计矩阵乘法和 LLM 推理等工作负载的分区功耗。

摘要:

云数据中心中的高效功耗管理对于降低成本、提升性能并最小化环境影响至关重要。GPU 对机器学习(ML)和 GenAI 等任务至关重要,同时也是功耗的主要来源。NVIDIA 的 Multi-Instance GPU(MIG)技术通过支持隔离分区以及按分区进行资源跟踪,提高了 GPU 利用率,促进了多个租户共享 GPU。然而,由于缺乏硬件支持,在 MIG 实例之间准确分摊 GPU 功耗仍然具有挑战性。本文通过开发用于估计每个 MIG 分区功耗的软件方法来应对这一挑战。我们分析了 NVIDIA GPU 利用率指标,发现构建具有良好准确性的轻量级方法可能十分困难。因此,我们探索使用基于 ML 的功耗模型,以实现准确的分区级功耗估计。研究结果表明,单一通用的离线功耗模型或建模方法并不适用于多样化工作负载,尤其是在并发 MIG 使用情况下;而利用正在执行工作负载的分区级利用率指标构建的在线模型,能够显著提升准确性。我们使用 NVIDIA A100 GPU,针对矩阵乘法和大语言模型推理等工作负载演示了该方法在准确分区级功耗估计方面的应用,为透明且公平的碳排放报告作出贡献。

英文摘要:

Efficient power management in cloud data centers is essential for reducing costs, enhancing performance, and minimizing environmental impact. GPUs, critical for tasks like machine learning (ML) and GenAI, are major contributors to power consumption. NVIDIA's Multi-Instance GPU (MIG) technology improves GPU utilization by enabling isolated partitions with per-partition resource tracking, facilitating GPU sharing by multiple tenants. However, accurately apportioning GPU power consumption among MIG instances remains challenging due to a lack of hardware support. This paper addresses this challenge by developing software methods to estimate power usage per MIG partition. We analyze NVIDIA GPU utilization metrics and find that light-weight methods with good accuracy can be difficult to construct. We hence explore the use of ML-based power models to enable accurate, partition-level power estimation. Our findings reveal that a single generic offline power model or modeling method is not applicable across diverse workloads, especially with concurrent MIG usage, and that online models constructed using partition-level utilization metrics of workloads under execution can significantly improve accuracy. Using NVIDIA A100 GPUs, we demonstrate this approach for accurate partition-level power estimation for workloads including matrix multiplication and Large Language Model inference, contributing to transparent and fair carbon reporting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑