使用Parquet数据集格式和混合精度训练改进机器学习回归模型的碳足迹——第二部分
Improve Machine Learning carbon footprint using Parquet dataset format and Mixed Precision training for regression models -- Part II
AI总结:
本文比较了CSV与Parquet格式及混合精度训练对回归模型功耗的影响,发现特定超参数下混合精度可降低功耗7-11瓦,但统计检验表明这些技术对整体碳足迹无显著改善。
摘要:
这是我的硕士学位论文的第二部分,比较了在训练回归机器学习模型时,使用逗号分隔值(CSV)和Parquet数据集格式以及默认浮点(32位)和Nvidia混合精度(16位和32位)的功耗。实验使用了与第一部分相同的定制PC(第一部分专用于分类测试和分析),并选择了不同的机器学习超参数(如批次大小、神经元数量和周期)来构建深度神经网络(DNN)。以默认超参数值的DNN基准测试作为参考,实验则使用了不同设置的组合。结果记录在Excel中,并选择描述性统计来计算各组间的平均值,通过图表进行比较。使用混合精度结合特定超参数时结果呈积极态势。与基准测试相比,优化回归模型使功耗降低了7至11瓦。回归结果表明,尽管混合精度有助于改善功耗,但必须谨慎考虑超参数。大批次大小和高神经元数量会对功耗产生负面影响。然而,本研究需要推断性统计,特别是方差分析(ANOVA)和T检验,来比较均值间的关系。结果报告称回归测试中的均值之间没有统计学显著性,并接受了H0。因此,选择不同的机器学习技术和Parquet数据集格式不会改善计算功耗和整体机器学习碳足迹。然而,在GPU集群上进行更广泛的实施可以显著增加样本量,因为这是一个关键因素,能够改变统计分析的结果。
英文摘要:
This is the 2nd part of the dissertation for my master degree and compared the power consumption using the Comma-Separated-Values (CSV) and parquet dataset format with the default floating point (32bit) and Nvidia mixed precision (16bit and 32bit) while training a regression ML model. The same custom PC as per the 1st part, which was dedicated to the classification testing and analysis, was built to perform the experiments, and different ML hyper-parameters, such as batch size, neurons, and epochs, were chosen to build Deep Neural Networks (DNN). A benchmarking test with default hyper-parameter values for the DNN was used as a reference, while the experiments used a combination of different settings. The results were recorded in Excel, and descriptive statistics were chosen to calculate the mean between the groups and compare them using graphs and tables. The outcome was positive when using mixed precision combined with specific hyper-parameters. Compared to the benchmarking, optimising the regression models reduced the power consumption between 7 and 11 Watts. The regression results show that while mixed precision can help improve power consumption, we must carefully consider the hyper-parameters. A high number of batch sizes and neurons will negatively affect power consumption. However, this research required inferential statistics, specifically ANOVA and T-test, to compare the relationship between the means. The results reported no statistical significance between the means in the regression tests and accepted H0. Therefore, choosing different ML techniques and the Parquet dataset format will not improve the computational power consumption and the overall ML carbon footprint. However, a more extensive implementation with a cluster of GPUs can increase the sample size significantly, as it is an essential factor and can change the outcome of the statistical analysis.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。