arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.20985cs.LGcs.AI

基于双向门控循环单元优化Transformer的深度学习任务GPU内存需求预测

GPU Memory Requirement Prediction for Deep Learning Task Based on Bidirectional Gated Recurrent Unit Optimization Transformer

Rice University(莱斯大学);Duke University(杜克大学)

Chao Wang, Zhizhao Wen, Ruoxin Zhang, Puyang Xu, Yifan Jiang

AI总结:

针对深度学习GPU内存预测难题,提出BiGRU优化Transformer模型,在MSE、RMSE等指标上优于决策树等基准方法,显著提升预测精度,助力资源调度。

摘要:

针对深度学习任务中对GPU内存资源准确预测的需求日益迫切的问题,本文深入分析了当前研究现状,创新性地提出了一种融合双向门控循环单元(BiGRU)来优化Transformer架构的深度学习模型,旨在提高内存需求预测的准确性。为验证模型的有效性,精心设计了对比实验,选取了四种具有代表性的基础机器学习模型:决策树、随机森林、Adaboost和XGBoost作为基准。详细的实验结果表明,本文提出的BiGRU Transformer优化模型在关键评价指标上展现出显著优势:在均方误差(MSE)和均方根误差(RMSE)方面,该模型在所有对比模型中取得了最低值,其预测结果与实际值的偏差最小;在平均绝对误差(MAE)和决定系数(R2)指标上,该模型同样表现良好,结果均衡稳定,综合预测性能远超所对比的基准机器学习方法。综上所述,本研究成功构建的基于双向门控循环单元优化的Transformer模型能够高效、准确地完成深度学习任务中的GPU内存需求预测任务,其预测精度相比传统机器学习方法有显著提升。本研究为优化深度学习任务的资源调度与管理、提高计算集群的利用效率提供了有力的技术支持和可靠的理论依据。

英文摘要:

In response to the increasingly critical demand for accurate prediction of GPU memory resources in deep learning tasks, this paper deeply analyzes the current research status and innovatively proposes a deep learning model that integrates bidirectional gated recurrent units (BiGRU) to optimize the Transformer architecture, aiming to improve the accuracy of memory demand prediction. To verify the effectiveness of the model, a carefully designed comparative experiment was conducted, selecting four representative basic machine learning models: decision tree, random forest, Adaboost, and XGBoost as benchmarks. The detailed experimental results show that the BiGRU Transformer optimization model proposed in this paper exhibits significant advantages in key evaluation indicators: in terms of mean square error (MSE) and root mean square error (RMSE), the model achieves the lowest value among all comparison models, and its predicted results have the smallest deviation from the actual values; In terms of mean absolute error (MAE) and coefficient of determination (R2) indicators, the model also performs well and the results are balanced and stable, with comprehensive predictive performance far exceeding the benchmark machine learning methods compared. In summary, the Transformer model based on bidirectional gated recurrent unit optimization successfully constructed in this study can efficiently and accurately complete GPU memory demand prediction tasks in deep learning tasks, and its prediction accuracy has been significantly improved compared to traditional machine learning methods. This research provides strong technical support and reliable theoretical basis for optimizing resource scheduling and management of deep learning tasks, and improving the utilization efficiency of computing clusters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑