arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.08330cs.PFcs.AIcs.ARcs.LG
收藏

一切皆关乎 PR —— 使用性能代表对 AI 加速器进行智能基准测试

It's all about PR -- Smart Benchmarking AI Accelerators using Performance Representatives

Embedded Systems, University of Tübingen(嵌入式系统,图宾根大学)

Alexander Louis-Ferdinand Jung, Jannik Steinmetz, Jonathan Gietz, Konstantin Lübeck, Oliver Bringmann

AI总结:

本文提出一种利用性能代表(PR)对AI加速器进行智能基准测试的方法,通过目标架构知识和参数扫描识别DNN层的PR以减少训练样本并提高性能估计精度。

摘要:

统计模型被广泛用于估算商用现货(COTS)AI硬件加速器的性能。然而,统计性能模型的训练通常需要大量数据,导致巨大的时间投入,并且在硬件可用性有限的情况下可能很困难。为了缓解这一问题,本文提出了一种新颖的性能建模方法,该方法在保持良好精度的同时显著减少了训练样本的数量。我们的方法利用对目标硬件架构的了解和初始参数扫描来识别一组性能代表(PR)以对深度神经网络(DNN)层进行基准测试。这些PR用于对统计性能模型进行基准测试并做出估计。与随机采样相比,这种有针对性的方法显著减少了所需的训练样本数量,同时实现了更好的估计精度。对于单层估计,我们实现了低至0.02%的平均绝对百分比误差(MAPE),而对于整个DNN估计,使用不到10000个训练样本实现了0.78%的误差。结果表明,在单层估计方面,与使用相同规模的随机采样数据集训练的模型相比,我们的方法具有优越性。

英文摘要:

Statistical models are widely used to estimate the performance of commercial off-the-shelf (COTS) AI hardware accelerators. However, training of statistical performance models often requires vast amounts of data, leading to a significant time investment and can be difficult in case of limited hardware availability. To alleviate this problem, we propose a novel performance modeling methodology that significantly reduces the number of training samples while maintaining good accuracy. Our approach leverages knowledge of the target hardware architecture and initial parameter sweeps to identify a set of Performance Representatives (PR) for deep neural network (DNN) layers. These PRs are then used for benchmarking, building a statistical performance model, and making estimations. This targeted approach drastically reduces the number of training samples needed, opposed to random sampling, to achieve a better estimation accuracy. We achieve a Mean Absolute Percentage Error (MAPE) of as low as 0.02% for single-layer estimations and 0.68% for whole DNN estimations with less than 10000 training samples. The results demonstrate the superiority of our method for single-layer estimations compared to models trained with randomly sampled datasets of the same size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑