arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2007.04452cs.AIcs.LG
收藏

NASGEM:基于图嵌入方法的神经网络结构搜索

NASGEM: Neural Architecture Search via Graph Embedding Method

Duke University(杜克大学);Facebook, Inc(脸书公司);Tsinghua University(清华大学);University of Nevada, Reno(内华达大学雷诺分校)

Hsin-Pai Cheng, Tunhou Zhang, Yixing Zhang, Shiyu Li, Feng Liang, Feng Yan, Meng Li, Vikas Chandra, Hai Li, Yiran Chen

摘要:

神经结构搜索(NAS)使神经网络的设计自动化并繁荣起来。最近提出了基于估计器的NAS来建模架构与其性能之间的关系,以实现可扩展和灵活的搜索。然而,现有的基于估计的方法编码的架构到一个潜在的空间,而不考虑图形的相似性。忽略基于节点的搜索空间中的图相似性可能引起相似图之间的大的不一致性以及它们在连续编码空间中的距离,导致不准确的编码表示和/或降低的表示容量,这可能产生次优的搜索结果。为了在编码中保留图的相关信息,我们提出了NASGEM,它代表通过图嵌入方法的神经结构搜索。NASGEM是由一种新的图嵌入方法,配备了相似性措施,捕捉图的拓扑信息。NASGEM通过精确估计图的距离并使用辅助Weisfeiler-Lehman核来指导编码,可以利用额外的结构信息来获得更精确的图表示,从而提高搜索效率。GEMNet是NASGEM发现的一组网络,在分类任务中始终优于现有搜索方法制作的网络,即,精度提高0.4%-3.6%,而乘法累积次数减少11%- 21%。我们进一步转移GEMNet用于COCO目标检测。在单阶段和两阶段检测器中,我们的GEMNet都超过了手动制作和自动搜索的同类产品。

英文摘要:

Neural Architecture Search (NAS) automates and prospers the design of neural networks. Estimator-based NAS has been proposed recently to model the relationship between architectures and their performance to enable scalable and flexible search. However, existing estimator-based methods encode the architecture into a latent space without considering graph similarity. Ignoring graph similarity in node-based search space may induce a large inconsistency between similar graphs and their distance in the continuous encoding space, leading to inaccurate encoding representation and/or reduced representation capacity that can yield sub-optimal search results. To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method. NASGEM is driven by a novel graph embedding method equipped with similarity measures to capture the graph topology information. By precisely estimating the graph distance and using an auxiliary Weisfeiler-Lehman kernel to guide the encoding, NASGEM can utilize additional structural information to get more accurate graph representation to improve the search efficiency. GEMNet, a set of networks discovered by NASGEM, consistently outperforms networks crafted by existing search methods in classification tasks, i.e., with 0.4%-3.6% higher accuracy while having 11%- 21% fewer Multiply-Accumulates. We further transfer GEMNet for COCO object detection. In both one-stage and twostage detectors, our GEMNet surpasses its manually-crafted and automatically-searched counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑