arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.07191cs.LGstat.ML
收藏

基于贝叶斯优化和最优传输的神经结构搜索

Neural Architecture Search with Bayesian Optimisation and Optimal Transport

Carnegie Mellon University(卡内基梅隆大学);Petuum Inc(Petuum公司)

Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing

摘要:

贝叶斯优化(BO)是指一类用于全局优化函数$f$的方法,该函数只能通过点评估来访问。它通常用于评估$f$成本较高的环境中。BO在机器学习中的一个常见用例是模型选择,在这种情况下,不可能对统计模型的泛化性能进行分析建模,我们求助于噪声和昂贵的训练和验证过程来选择最佳模型。传统的BO方法集中在欧几里得域和分类域,在模型选择的背景下,仅允许调整机器学习算法的标量超参数。然而,随着人们对深度学习的兴趣激增,对神经网络\emph{architectures}进行调优的需求也越来越大。在这项工作中,我们开发了NASBOT,一个基于高斯过程的BO神经结构搜索框架。为了实现这一点,我们开发了一个距离度量空间的神经网络架构,可以有效地计算通过一个最佳的运输计划。这个距离可能是深度学习社区的独立兴趣,因为它可能会在BO之外找到应用程序。我们证明,NASBOT优于其他替代方案的架构搜索在几个交叉验证的多层感知器和卷积神经网络的模型选择任务。

英文摘要:

Bayesian Optimisation (BO) refers to a class of methods for global optimisation of a function $f$ which is only accessible via point evaluations. It is typically used in settings where $f$ is expensive to evaluate. A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model. Conventional BO methods have focused on Euclidean and categorical domains, which, in the context of model selection, only permits tuning scalar hyper-parameters of machine learning algorithms. However, with the surge of interest in deep learning, there is an increasing demand to tune neural network \emph{architectures}. In this work, we develop NASBOT, a Gaussian process based BO framework for neural architecture search. To accomplish this, we develop a distance metric in the space of neural network architectures which can be computed efficiently via an optimal transport program. This distance might be of independent interest to the deep learning community as it may find applications outside of BO. We demonstrate that NASBOT outperforms other alternatives for architecture search in several cross validation based model selection tasks on multi-layer perceptrons and convolutional neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑