arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.07554cs.LGcs.AI
收藏

高效的架构搜索,满足不同的任务需求

Efficient Architecture Search for Diverse Tasks

Carnegie Mellon University(卡内基梅隆大学)

Junhong Shen, Mikhail Khodak, Ameet Talwalkar

摘要:

虽然神经架构搜索(NAS)已经为深入研究的领域提供了自动化机器学习(AutoML),但其在计算机视觉之外的任务中的应用仍处于探索阶段。由于较少研究的领域正是我们期望AutoML产生最大影响的领域,因此在这项工作中,我们研究NAS以有效地解决各种问题。为了寻求一种快速、简单和广泛适用的方法,我们修复了标准卷积网络(CNN)拓扑,并提出搜索其操作应采用的正确内核大小和膨胀,这大大扩展了模型在不同类型数据的多分辨率下提取特征的能力,同时只需要在操作空间上搜索。为了克服在这个搜索空间中朴素权重共享的效率挑战,我们引入了DASH,这是一种可微分的NAS算法,使用卷积的傅立叶对角化来计算混合运算,在实践中实现了更好的渐近复杂度和高达10倍的搜索时间加速。我们评估了DASH在跨越各种应用领域的10个任务,如PDE求解,蛋白质折叠和心脏病检测。DASH在总体上优于最先进的AutoML方法,在七个任务上实现了最知名的自动化性能。与此同时,在十个任务中的六个任务上,搜索和再训练的组合时间比简单地训练一个准确度低得多的CNN主干慢不到2倍。

英文摘要:

While neural architecture search (NAS) has enabled automated machine learning (AutoML) for well-researched areas, its application to tasks beyond computer vision is still under-explored. As less-studied domains are precisely those where we expect AutoML to have the greatest impact, in this work we study NAS for efficiently solving diverse problems. Seeking an approach that is fast, simple, and broadly applicable, we fix a standard convolutional network (CNN) topology and propose to search for the right kernel sizes and dilations its operations should take on. This dramatically expands the model's capacity to extract features at multiple resolutions for different types of data while only requiring search over the operation space. To overcome the efficiency challenges of naive weight-sharing in this search space, we introduce DASH, a differentiable NAS algorithm that computes the mixture-of-operations using the Fourier diagonalization of convolution, achieving both a better asymptotic complexity and an up-to-10x search time speedup in practice. We evaluate DASH on ten tasks spanning a variety of application domains such as PDE solving, protein folding, and heart disease detection. DASH outperforms state-of-the-art AutoML methods in aggregate, attaining the best-known automated performance on seven tasks. Meanwhile, on six of the ten tasks, the combined search and retraining time is less than 2x slower than simply training a CNN backbone that is far less accurate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑