黑箱优化算法最优解的GPU加速穷举搜索
GPU Accelerated Exhaustive Search for Optimal Ensemble of Black-Box Optimization Algorithms
摘要：
黑盒优化对于调整复杂的机器学习算法至关重要，这些算法更容易实验而不是理解。在本文中，我们表明，一个简单的集成的黑盒优化算法可以优于任何一个。然而，寻找这样的最佳合奏需要大量的实验。我们提出了一个多GPU优化的框架，通过并行运行许多实验来加速暴力搜索黑盒优化算法的最佳集成。轻量级优化由CPU执行，而昂贵的模型训练和评估则分配给GPU。我们通过训练270万个模型和运行541，440次优化来评估15个优化器。在DGX-1上，搜索时间从两个20核CPU上的10多天减少到8核GPU上的不到24小时。通过GPU加速的穷举搜索找到的最佳集成，我们赢得了NeurIPS 2020黑盒优化挑战赛的第二名。
英文摘要：
Black-box optimization is essential for tuning complex machine learning algorithms which are easier to experiment with than to understand. In this paper, we show that a simple ensemble of black-box optimization algorithms can outperform any single one of them. However, searching for such an optimal ensemble requires a large number of experiments. We propose a Multi-GPU-optimized framework to accelerate a brute force search for the optimal ensemble of black-box optimization algorithms by running many experiments in parallel. The lightweight optimizations are performed by CPU while expensive model training and evaluations are assigned to GPUs. We evaluate 15 optimizers by training 2.7 million models and running 541,440 optimizations. On a DGX-1, the search time is reduced from more than 10 days on two 20-core CPUs to less than 24 hours on 8-GPUs. With the optimal ensemble found by GPU-accelerated exhaustive search, we won the 2nd place of NeurIPS 2020 black-box optimization challenge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
