通过学习从数据集生成图来快速搜索神经结构
Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets
摘要：
尽管最近的神经架构搜索（NAS）方法在各种任务上取得了成功，这些任务已经显示出输出的网络在很大程度上优于人类设计的网络，但传统的NAS方法主要解决了搜索单个任务（数据集）的网络架构的优化问题，这在多个任务（数据集）之间并没有很好地推广。此外，由于这种特定于任务的方法从零开始为每个给定的任务搜索神经架构，因此它们产生了很大的计算成本，当时间和金钱预算有限时，这是有问题的。在本文中，我们提出了一个高效的NAS框架，该框架在由数据集和预训练网络组成的数据库上训练一次，并且可以快速搜索新数据集的神经架构。提出的MetaD 2A（Meta Dataset-to-Architecture）模型可以通过使用摊销元学习学习的跨模态潜在空间从给定集合（数据集）随机生成图（架构）。此外，我们还提出了一个元性能预测器来估计和选择最佳架构，而无需直接对目标数据集进行训练。实验结果表明，我们的模型在NAS-Bench 201搜索空间的ImageNet-1 K子集和架构上进行了元学习，成功地推广到了多个看不见的数据集，包括CIFAR-10和CIFAR-100，平均搜索时间为33 GPU秒。即使在MobileNetV 3搜索空间下，MetaD 2A也比可转移NAS方法NSGANetV 2快5.5K倍，性能相当。我们相信MetaD 2A为快速NAS提出了一个新的研究方向，以及如何利用过去几年积累的丰富数据集和架构数据库中的知识。代码可在https://github.com/HayeonLee/MetaD2A上获得。
英文摘要：
Despite the success of recent Neural Architecture Search (NAS) methods on various tasks which have shown to output networks that largely outperform human-designed networks, conventional NAS methods have mostly tackled the optimization of searching for the network architecture for a single task (dataset), which does not generalize well across multiple tasks (datasets). Moreover, since such task-specific methods search for a neural architecture from scratch for every given task, they incur a large computational cost, which is problematic when the time and monetary budget are limited. In this paper, we propose an efficient NAS framework that is trained once on a database consisting of datasets and pretrained networks and can rapidly search for a neural architecture for a novel dataset. The proposed MetaD2A (Meta Dataset-to-Architecture) model can stochastically generate graphs (architectures) from a given set (dataset) via a cross-modal latent space learned with amortized meta-learning. Moreover, we also propose a meta-performance predictor to estimate and select the best architecture without direct training on target datasets. The experimental results demonstrate that our model meta-learned on subsets of ImageNet-1K and architectures from NAS-Bench 201 search space successfully generalizes to multiple unseen datasets including CIFAR-10 and CIFAR-100, with an average search time of 33 GPU seconds. Even under MobileNetV3 search space, MetaD2A is 5.5K times faster than NSGANetV2, a transferable NAS method, with comparable performance. We believe that the MetaD2A proposes a new research direction for rapid NAS as well as ways to utilize the knowledge from rich databases of datasets and architectures accumulated over the past years. Code is available at https://github.com/HayeonLee/MetaD2A.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
