arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.08666cs.CV
收藏

再谈免训练的NAS学习算法:一种有效的基于训练的方法

Revisiting Training-free NAS Metrics: An Efficient Training-based Method

University of Central Florida(中佛罗里达大学);Bytedance Inc.(字节跳动公司)

Taojiannan Yang, Linjie Yang, Xiaojie Jin, Chen Chen

摘要:

最近的神经结构搜索(NAS)工作提出了训练的免费指标来排名网络,大大降低了搜索成本的NAS。在本文中,我们重新审视了这些无训练指标,并发现:(1)参数的数量(\#Param),这是最直接的无训练指标,在以前的工作中被忽视,但令人惊讶的是有效的,(2)最近的无训练指标在很大程度上依赖于\#Param信息来排名网络。我们的实验表明,当\#Param信息不可用时,最近的无训练指标的性能急剧下降。受这些观察的启发,我们认为,与\#Param相关性较低的指标是希望为NAS提供额外的信息。我们提出了一个轻量级的基于训练的度量,它与\#Param具有弱相关性,同时以较低的搜索成本实现比无训练度量更好的性能。具体而言,在DARTS搜索空间上,我们的方法仅用2.6 GPU小时就完成了直接在ImageNet上的搜索,并实现了24.1\%/7.1\%的top-1/top-5出错率,这在最先进的NAS方法中具有竞争力。代码可从\url{https://github.com/taoyang1122/Revisit_TrainingFree_NAS}获得

英文摘要:

Recent neural architecture search (NAS) works proposed training-free metrics to rank networks which largely reduced the search cost in NAS. In this paper, we revisit these training-free metrics and find that: (1) the number of parameters (\#Param), which is the most straightforward training-free metric, is overlooked in previous works but is surprisingly effective, (2) recent training-free metrics largely rely on the \#Param information to rank networks. Our experiments show that the performance of recent training-free metrics drops dramatically when the \#Param information is not available. Motivated by these observations, we argue that metrics less correlated with the \#Param are desired to provide additional information for NAS. We propose a light-weight training-based metric which has a weak correlation with the \#Param while achieving better performance than training-free metrics at a lower search cost. Specifically, on DARTS search space, our method completes searching directly on ImageNet in only 2.6 GPU hours and achieves a top-1/top-5 error rate of 24.1\%/7.1\%, which is competitive among state-of-the-art NAS methods. Codes are available at \url{https://github.com/taoyang1122/Revisit_TrainingFree_NAS}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑