arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.09128cs.LGstat.ML
收藏

DSNAS:无需参数再训练的直接神经结构搜索

DSNAS: Direct Neural Architecture Search without Parameter Retraining

University of California, Los Angeles(加州大学洛杉矶分校);Cambridge University(剑桥大学);SenseTime Research(商汤科技研究院)

Shoukang Hu, Sirui Xie, Hehui Zheng, Chunxiao Liu, Jianping Shi, Xunying Liu, Dahua Lin

摘要:

如果NAS方法是解决方案,那么问题是什么?大多数现有的NAS方法需要两个阶段的参数优化。然而,同一架构在这两个阶段的性能相关性很差。在这项工作中,我们提出了一个新的问题定义为NAS,特定于任务的端到端,根据这一观察。我们认为,给定期望NAS方法的计算机视觉任务,该定义可以将严格定义的NAS评估减少到i)该任务的准确性和ii)最终获得具有令人满意的准确性的模型所消耗的总计算。看到大多数现有的方法不直接解决这个问题,我们提出DSNAS,一个有效的可微分NAS框架,同时优化架构和参数与低偏蒙特卡罗估计。从DSNAS派生的子网络可以直接部署,而无需参数重新训练。与两阶段方法相比,DSNAS在ImageNet上成功发现了具有相当准确度(74.4%)的网络,耗时420 GPU小时,总时间减少了34%以上。我们的实现可以在https://github.com/SNAS-Series/SNAS-Series上获得。

英文摘要:

If NAS methods are solutions, what is the problem? Most existing NAS methods require two-stage parameter optimization. However, performance of the same architecture in the two stages correlates poorly. In this work, we propose a new problem definition for NAS, task-specific end-to-end, based on this observation. We argue that given a computer vision task for which a NAS method is expected, this definition can reduce the vaguely-defined NAS evaluation to i) accuracy of this task and ii) the total computation consumed to finally obtain a model with satisfying accuracy. Seeing that most existing methods do not solve this problem directly, we propose DSNAS, an efficient differentiable NAS framework that simultaneously optimizes architecture and parameters with a low-biased Monte Carlo estimate. Child networks derived from DSNAS can be deployed directly without parameter retraining. Comparing with two-stage methods, DSNAS successfully discovers networks with comparable accuracy (74.4%) on ImageNet in 420 GPU hours, reducing the total time by more than 34%. Our implementation is available at https://github.com/SNAS-Series/SNAS-Series.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑