arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.13577cs.DC
收藏

基于图像的GPU核自动调整搜索技术分析:样本大小的影响

Analyzing Search Techniques for Autotuning Image-based GPU Kernels: The Impact of Sample Sizes

Jacob O. Tørring, Anne C. Elster

摘要:

现代计算系统越来越复杂,其多核CPU和GPU加速器每年都在变化,甚至更频繁。因此,编写有效使用相关复杂存储器系统并利用可用并行性的程序变得非常具有挑战性。自动调优通过搜索最佳参数集将参数化代码优化到目标硬件来解决这个问题。因此,在过去的几十年中,经验自整定获得了兴趣。虽然新的自动调整算法定期提出和出版,我们将展示为什么比较这些自动调整算法是一个看似困难的任务。在本文中,我们描述了我们的实证研究的国家的最先进的搜索技术进行自动调整,通过比较它们的样本量,基准和架构的范围。我们优化了6个可调参数,搜索空间大小超过200万。所研究的算法包括随机搜索(RS)、随机森林回归(RF)、遗传算法(GA)、高斯过程贝叶斯优化(BO GP)和树-Parzen估计贝叶斯优化(BO TPE)。我们在ImageCL基准测试套件上的结果表明,理想的自动调整算法在很大程度上取决于样本大小。在我们的研究中,BO GP和BO TPE在样本量从25到100的大多数情况下都优于其他算法。然而,遗传算法通常优于其他的样本大小200及以上。我们通常会看到在样本大小的较低范围(25-100)中,RS的加速效果最好。然而,对于更高的样本量(200-400),算法的性能更一致地优于RS。因此,对于所有样本量,没有一种最先进的算法优于其他算法。对今后的工作也提出了一些建议。

英文摘要:

Modern computing systems are increasingly more complex, with their multicore CPUs and GPUs accelerators changing yearly, if not more often. It thus has become very challenging to write programs that efficiently use the associated complex memory systems and take advantage of the available parallelism. Autotuning addresses this by optimizing parameterized code to the targeted hardware by searching for the optimal set of parameters. Empirical autotuning has therefore gained interest during the past decades. While new autotuning algorithms are regularly presented and published, we will show why comparing these autotuning algorithms is a deceptively difficult task. In this paper, we describe our empirical study of state-of-the-art search techniques for autotuning by comparing them on a range of sample sizes, benchmarks and architectures. We optimize 6 tunable parameters with a search-space size of over 2 million. The algorithms studied include Random Search (RS), Random Forest Regression (RF), Genetic Algorithms (GA), Bayesian Optimization with Gaussian Processes (BO GP) and Bayesian Optimization with Tree-Parzen Estimators (BO TPE). Our results on the ImageCL benchmark suite suggest that the ideal autotuning algorithm heavily depends on the sample size. In our study, BO GP and BO TPE outperform the other algorithms in most scenarios with sample sizes from 25 to 100. However, GA usually outperforms the others for sample sizes 200 and beyond. We generally see the most speedup to be gained over RS in the lower range of sample sizes (25-100). However, the algorithms more consistently outperform RS for higher sample sizes (200-400). Hence, no single state-of-the-art algorithm outperforms the rest for all sample sizes. Some suggestions for future work are also included.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑