监督超参数配置的多保真竞赛
Supervising the Multi-Fidelity Race of Hyperparameter Configurations
AI总结：
针对多保真HPO预算分配次优问题，提出DyHPO贝叶斯优化方法，利用嵌入学习曲线动态的GP深度核与多预算采集函数动态决定配置训练，并在50个数据集和多类架构上验证其显著优势。
摘要：
多保真（灰盒）超参数优化技术（HPO）近来已成为调优深度学习方法的一个有前景方向。然而，现有方法存在将HPO预算次优地分配给超参数配置的问题。在本工作中，我们提出DyHPO，这是一种贝叶斯优化方法，能够学习决定在所有可行配置之间的动态竞赛中进一步训练哪个超参数配置。我们提出了一种用于高斯过程的新型深度核，可嵌入学习曲线动态，并提出了一种融合多预算信息的采集函数。通过涵盖50个数据集（表格、图像、NLP）和多种架构（MLP、CNN/NAS、RNN）的大规模实验，我们证明了DyHPO相对于最先进超参数优化方法的显著优越性。
英文摘要：
Multi-fidelity (gray-box) hyperparameter optimization techniques (HPO) have recently emerged as a promising direction for tuning Deep Learning methods. However, existing methods suffer from a sub-optimal allocation of the HPO budget to the hyperparameter configurations. In this work, we introduce DyHPO, a Bayesian Optimization method that learns to decide which hyperparameter configuration to train further in a dynamic race among all feasible configurations. We propose a new deep kernel for Gaussian Processes that embeds the learning curve dynamics, and an acquisition function that incorporates multi-budget information. We demonstrate the significant superiority of DyHPO against state-of-the-art hyperparameter optimization methods through large-scale experiments comprising 50 datasets (Tabular, Image, NLP) and diverse architectures (MLP, CNN/NAS, RNN).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
