arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.09259stat.MLcs.LGmath.OC
收藏

考虑公共随机数的贝叶斯优化

Bayesian Optimization Allowing for Common Random Numbers

University of Warwick(华威大学);Uber AI(优步人工智能);University of Arizona(亚利桑那大学)

Michael Pearce, Matthias Poloczek, Juergen Branke

摘要:

贝叶斯优化是一个强大的工具,昂贵的随机黑箱优化问题,如基于模拟的优化或机器学习超参数调整。许多随机目标函数隐含地需要随机数种子作为输入。通过显式重用种子,用户可以利用公共随机数,在相同的随机生成的场景下比较两个或更多个输入,例如作业车间问题中的公共客户流,或者将训练数据随机划分为用于机器学习算法的训练和验证集。为了找到一个在无限多个种子上具有最佳平均性能的输入,我们提出了一种新的高斯过程模型,该模型联合建模每个种子的输出和平均值。然后,我们介绍了知识梯度的共同随机数,迭代地确定输入和随机种子的组合,以评估的目标和自动权衡重用旧种子和查询新的种子,从而克服了需要评估批量输入或测量的差异对建议在以前的方法。我们研究了知识梯度的共同随机数的理论和经验,发现它实现了显着的性能改善,只有适度增加的计算成本。

英文摘要:

Bayesian optimization is a powerful tool for expensive stochastic black-box optimization problems such as simulation-based optimization or machine learning hyperparameter tuning. Many stochastic objective functions implicitly require a random number seed as input. By explicitly reusing a seed a user can exploit common random numbers, comparing two or more inputs under the same randomly generated scenario, such as a common customer stream in a job shop problem, or the same random partition of training data into training and validation set for a machine learning algorithm. With the aim of finding an input with the best average performance over infinitely many seeds, we propose a novel Gaussian process model that jointly models both the output for each seed and the average. We then introduce the Knowledge gradient for Common Random Numbers that iteratively determines a combination of input and random seed to evaluate the objective and automatically trades off reusing old seeds and querying new seeds, thus overcoming the need to evaluate inputs in batches or measuring differences of pairs as suggested in previous methods. We investigate the Knowledge Gradient for Common Random Numbers both theoretically and empirically, finding it achieves significant performance improvements with only moderate added computational cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑