arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.05933cs.PFcs.LG
收藏

用于资源受限嵌入式设备上小型语言模型微调的节能GPU动态电压频率缩放

Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices

School of Engineering and Digital Sciences, Nazarbayev University(工程与数字科学学院,纳扎尔拜耶夫大学)

Jurn-Gyu Park, Sanzhar Zholdybayev, Aidar Amangeldi, Ademi Zhanuzakova

AI总结:

研究资源受限嵌入式设备上SLM微调的节能问题,提出基于机器学习的模型选择方法来选GPU DVFS设置,在NVIDIA Jetson AGX Orin上实验,相比MAXN模式0平均节能13.11%,最高达26.73%。

摘要:

在资源受限的嵌入式GPU平台上,动态电压频率缩放(DVFS)对于节能的小型语言模型(SLM)微调至关重要。随着隐私和个性化驱动的适配越来越需要本地执行,且涉及对许多小批次进行反复的前向-后向优化,这使其比单通道推理更耗时且耗能。为此,首先对BERT变体的代表性仅编码器SLM和Pythia变体的仅自回归解码器SLM在GLUE基准上的微调行为进行了表征。此外,提出了一种简单而有效的基于机器学习的模型选择方法,可在资源受限的嵌入式平台上选择能量最优的GPU DVFS设置。在NVIDIA Jetson AGX Orin上的结果表明,与无明确功率上限的MAXN模式0相比,平均节能13.11%(最高可达26.73%)。

英文摘要:

Dynamic Voltage Frequency Scaling (DVFS) on resource-constrained embedded GPU platforms is essential for energy-efficient small language model (SLM) fine-tuning, as privacy- and personalization-driven adaptation increasingly requires local execution and involves repeated forward-backward optimization over many mini-batches, making it substantially more time- and energy-intensive than single-pass inference. To this end, 1) we first characterize the fine-tuning behavior of representative encoder-only SLMs of BERT variants, and autoregressive decoder-only SLMs of Pythia variants on GLUE benchmarks. In addition to the characterizations, 2) we propose a simple yet effective ML-based model selection that selects energy-optimal GPU DVFS settings on resource-constrained embedded platforms. Our results on NVIDIA Jetson AGX Orin demonstrate average 13.11% energy savings (up to 26.73%) over MAXN Mode 0, which has no explicit power cap.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑