用于CPU后端的自动调优TensorFlow线程模型
Auto-tuning TensorFlow Threading Model for CPU Backend
摘要:
TensorFlow是一个流行的深度学习框架,数据科学家使用它来解决各种机器学习和深度学习问题,例如图像分类和语音识别。它还可以在大规模和异构环境中运行-它允许用户训练神经网络模型或使用GPU、CPU和深度学习特定定制设计的硬件(如TPU)部署它们进行推理。尽管TensorFlow支持各种优化的后端,但使用后端实现最佳性能可能需要额外的努力。例如,从CPU后端获得最佳性能需要仔细调优其线程模型。不幸的是,目前使用的最佳调优方法是手动的、繁琐的、耗时的,而且更重要的是,可能无法保证最佳性能。在本文中,我们开发了一种名为TensorTuner的自动方法,用于搜索CPU后端TensorFlow线程模型的最佳参数设置。我们使用TensorFlow基准测试套件中的一组神经网络在Eigen和英特尔的MKL CPU后端上评估TensorTuner。我们的评估结果表明,TensorTuner发现的参数设置为Eigen CPU后端带来了2%至123%的性能提升,为MKL CPU后端带来了1.5%至28%的性能提升,而这些性能都是使用其最知名的参数设置获得的。这突出了Eigen CPU后端中的默认参数设置不是理想设置的事实;即使对于仔细手动调优的MKL后端,设置也可能是次优的。我们的评估还显示,TensorTuner在寻找最佳设置方面非常有效--它能够通过修剪超过90%的参数搜索空间来快速收敛到最佳设置。
英文摘要:
TensorFlow is a popular deep learning framework used by data scientists to solve a wide-range of machine learning and deep learning problems such as image classification and speech recognition. It also operates at a large scale and in heterogeneous environments --- it allows users to train neural network models or deploy them for inference using GPUs, CPUs and deep learning specific custom-designed hardware such as TPUs. Even though TensorFlow supports a variety of optimized backends, realizing the best performance using a backend may require additional efforts. For instance, getting the best performance from a CPU backend requires careful tuning of its threading model. Unfortunately, the best tuning approach used today is manual, tedious, time-consuming, and, more importantly, may not guarantee the best performance. In this paper, we develop an automatic approach, called TensorTuner, to search for optimal parameter settings of TensorFlow's threading model for CPU backends. We evaluate TensorTuner on both Eigen and Intel's MKL CPU backends using a set of neural networks from TensorFlow's benchmarking suite. Our evaluation results demonstrate that the parameter settings found by TensorTuner produce 2% to 123% performance improvement for the Eigen CPU backend and 1.5% to 28% performance improvement for the MKL CPU backend over the performance obtained using their best-known parameter settings. This highlights the fact that the default parameter settings in Eigen CPU backend are not the ideal settings; and even for a carefully hand-tuned MKL backend, the settings may be sub-optimal. Our evaluations also revealed that TensorTuner is efficient at finding the optimal settings --- it is able to converge to the optimal settings quickly by pruning more than 90% of the parameter search space.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。