DrNAS：Dirichlet神经结构搜索
DrNAS: Dirichlet Neural Architecture Search
摘要：
本文提出了一种新的可微结构搜索方法，将其转化为一个分布学习问题。我们把连续松弛的结构混合权重作为随机变量，用Dirichlet分布建模。利用最近开发的路径导数，可以很容易地用基于梯度的优化器以端到端的方式优化狄利克雷参数。该公式提高了泛化能力，并诱导随机性，自然鼓励在搜索空间中的探索。此外，为了减轻可区分NAS的大内存消耗，我们提出了一个简单而有效的渐进式学习方案，可以直接搜索大规模任务，消除搜索和评估阶段之间的差距。大量的实验证明了我们的方法的有效性。具体来说，在移动环境下，CIFAR-10的测试误差为2.46%，ImageNet为23.7%。在NAS-Bench-201上，我们还在所有三个数据集上获得了最先进的结果，并为神经架构搜索算法的有效设计提供了见解。
英文摘要：
This paper proposes a novel differentiable architecture search method by formulating it into a distribution learning problem. We treat the continuously relaxed architecture mixing weight as random variables, modeled by Dirichlet distribution. With recently developed pathwise derivatives, the Dirichlet parameters can be easily optimized with gradient-based optimizer in an end-to-end manner. This formulation improves the generalization ability and induces stochasticity that naturally encourages exploration in the search space. Furthermore, to alleviate the large memory consumption of differentiable NAS, we propose a simple yet effective progressive learning scheme that enables searching directly on large-scale tasks, eliminating the gap between search and evaluation phases. Extensive experiments demonstrate the effectiveness of our method. Specifically, we obtain a test error of 2.46% for CIFAR-10, 23.7% for ImageNet under the mobile setting. On NAS-Bench-201, we also achieve state-of-the-art results on all three datasets and provide insights for the effective design of neural architecture search algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
