arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.02488cs.LGcs.CV

Exploring Complicated Search Spaces with Interleaving-Free Sampling

University of Chinese Academy of Sciences(中国科学院大学);Huawei Inc.(华为公司);Huazhong University of Science and Technology(华中科技大学)

Yunjie Tian, Lingxi Xie, Jiemin Fang, Jianbin Jiao, Qixiang Ye, Qi Tian

英文摘要:

The existing neural architecture search algorithms are mostly working on search spaces with short-distance connections. We argue that such designs, though safe and stable, obstacles the search algorithms from exploring more complicated scenarios. In this paper, we build the search algorithm upon a complicated search space with long-distance connections, and show that existing weight-sharing search algorithms mostly fail due to the existence of \textbf{interleaved connections}. Based on the observation, we present a simple yet effective algorithm named \textbf{IF-NAS}, where we perform a periodic sampling strategy to construct different sub-networks during the search procedure, avoiding the interleaved connections to emerge in any of them. In the proposed search space, IF-NAS outperform both random sampling and previous weight-sharing search algorithms by a significant margin. IF-NAS also generalizes to the micro cell-based spaces which are much easier. Our research emphasizes the importance of macro structure and we look forward to further efforts along this direction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑