Modern computing systems are increasingly more complex, with their multicore CPUs and GPUs accelerators changing yearly, if not more often. It thus has become very challenging to write programs that efficiently use the associated complex memory systems and take advantage of the available parallelism. Autotuning addresses this by optimizing parameterized code to the targeted hardware by searching for the optimal set of parameters. Empirical autotuning has therefore gained interest during the past decades. While new autotuning algorithms are regularly presented and published, we will show why comparing these autotuning algorithms is a deceptively difficult task.
In this paper, we describe our empirical study of state-of-the-art search techniques for autotuning by comparing them on a range of sample sizes, benchmarks and architectures. We optimize 6 tunable parameters with a search-space size of over 2 million. The algorithms studied include Random Search (RS), Random Forest Regression (RF), Genetic Algorithms (GA), Bayesian Optimization with Gaussian Processes (BO GP) and Bayesian Optimization with Tree-Parzen Estimators (BO TPE).
Our results on the ImageCL benchmark suite suggest that the ideal autotuning algorithm heavily depends on the sample size. In our study, BO GP and BO TPE outperform the other algorithms in most scenarios with sample sizes from 25 to 100. However, GA usually outperforms the others for sample sizes 200 and beyond. We generally see the most speedup to be gained over RS in the lower range of sample sizes (25-100). However, the algorithms more consistently outperform RS for higher sample sizes (200-400). Hence, no single state-of-the-art algorithm outperforms the rest for all sample sizes. Some suggestions for future work are also included.
Analyzing Machine Learning Workloads Using a Detailed GPU Simulator
目前部署的大多数深度神经网络都是通过TensorFlow和PyTorch等高级框架使用GPU进行训练的。本文介绍了我们对GPGPU-Sim模拟器所做的更改,使其能够通过运行NVIDIA cuDNN库中包含的PTX内核来运行PyTorch。我们使用由此产生的修改后的模拟器(已随本文公开提供)来研究一些简单的深度学习工作负载。随着我们对GPGPU-Sim功能仿真模型的更改,我们发现GPGPU-Sim性能模型运行支持cuDNN的LeNet for MNIST实现报告结果在实际硬件的30%内。使用GPGPU-Sim的AerialVision性能分析工具,我们观察到cuDNN API调用包含许多不同的阶段,并且似乎包括潜在的低效微架构行为,例如DRAM分区银行露营,至少在GPGPU-Sim的当前性能模型上执行时。
FPGA or GPU? Analyzing comparative research for application-specific guidance
Western New England University, Member IEEE, Springfield, MA, USA(西部新英格兰大学);Department of ECE , The Citadel , Charleston, NC, USA(电子与计算机工程系,特 CITADEL,查尔斯顿,NC,USA)
A (Somewhat Dated) Comparative Study of Betweenness Centrality Algorithms on GPU
计算介数中心性(BC)的问题是很重要的分析图在许多实际应用中,如社交网络,生物网络,交通网络,电路等,因为这个问题是计算密集型,研究人员一直在开发算法,使用高性能的计算资源,如超级计算机,集群,图形处理单元(GPU)。当前用于计算BC的GPU算法采用Brandes的顺序算法,对于线程调度、数据结构和原子操作具有不同的折衷。在本文中,我们研究了三个GPU算法计算BC的未加权,有向,无标度网络。我们讨论和衡量他们的平衡线程调度,原子操作,同步和延迟隐藏的设计选择的权衡。我们的程序是用NVIDIA CUDA C编写的,并在NVIDIA Tesla M2050 GPU上测试。
成对序列比对是基因组数据分析中计算量最大的核心之一,占关键生物信息学应用运行时间的90%以上。由于分析长度在1 Kb和1 Mb之间的序列的高计算成本,这种方法对于第三代序列特别昂贵。鉴于精确配对算法的二次开销长alignment,社区主要依赖于近似算法,只搜索高质量的alignment,并在找不到时提前停止。在这项工作中,我们提出了流行的X-drop alignment算法的第一个GPU优化,我们命名为LOGAN。结果表明,我们的高性能多GPU实现高达181.6 GCUPS和速度高达6.6倍和30.7倍,分别使用1和6个NVIDIA Tesla V100,在最先进的软件上运行两个IBM Power 9处理器使用168 CPU线程,具有相同的精度。我们还证明了一个2.3倍的LOGAN速度比ksw 2,一个国家的最先进的矢量化算法的序列比对中实现minimap 2,一个长读映射软件。为了突出我们的工作对实际应用的影响,我们将LOGAN与一个名为BELLA的多对多长读段比对软件结合起来,并证明我们的实现将整体BELLA运行时间提高了10.6倍。最后,我们调整了LOGAN的Roofline模型,并证明我们的实现在NVIDIA Tesla V100上接近最佳。