Network spaces have been known as a critical factor in both handcrafted network designs or defining search spaces for Neural Architecture Search (NAS). However, an effective space involves tremendous prior knowledge and/or manual effort, and additional constraints are required to discover efficiency-aware architectures. In this paper, we define a new problem, Network Space Search (NSS), as searching for favorable network spaces instead of a single architecture. We propose an NSS method to directly search for efficient-aware network spaces automatically, reducing the manual effort and immense cost in discovering satisfactory ones. The resultant network spaces, named Elite Spaces, are discovered from Expanded Search Space with minimal human expertise imposed. The Pareto-efficient Elite Spaces are aligned with the Pareto front under various complexity constraints and can be further served as NAS search spaces, benefiting differentiable NAS approaches (e.g. In CIFAR-100, an averagely 2.3% lower error rate and 3.7% closer to target constraint than the baseline with around 90% fewer samples required to find satisfactory networks). Moreover, our NSS approach is capable of searching for superior spaces in future unexplored spaces, revealing great potential in searching for network spaces automatically. Website: https://minhungchen.netlify.app/publication/nss.
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets
South China University of Technology（华南理工大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Tencent（腾讯）；University of Adelaide（阿德莱德大学）；Pazhou Laboratory（琶洲实验室）
DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
National Tsing-Hua University（国立清华大学）；Google（谷歌）
神经架构搜索（NAS）的最新突破已经在图像分类和语言建模等应用中实现了最先进的性能。然而，这些技术通常忽略与设备相关的目标，例如推断时间、存储器使用和功耗。针对设备相关目标优化神经架构对于在计算资源有限的便携式设备上部署深度网络至关重要。我们提出了DPP-Net：用于Pareto最优神经架构的设备感知渐进式搜索，优化设备相关（例如，推断时间和存储器使用）和设备不可知（例如，精度和模型尺寸）目标。DPP-Net采用了受当前最先进的移动CNN启发的紧凑搜索空间，并通过采用渐进式搜索进一步提高了搜索效率（Liu et al. 2017）。在CIFAR-10上的实验结果将证明DPP-Net发现的Pareto最优网络在三种不同设备上的有效性：（1）使用Titan X GPU的工作站，（2）NVIDIA Jetson TX 1嵌入式系统，以及（3）使用ARM Cortex-A53的手机。与CondenseNet和NASNet（Mobile）相比，DPP-Net具有更好的性能：在各种设备上具有更高的准确性和更短的推理时间。额外的实验结果表明，DPP-Net发现的模型在ImageNet上也取得了非常好的性能。
Pareto-wise Ranking Classifier for Multi-objective Evolutionary Neural Architecture Search
College of Software, Northeastern University（东北大学软件学院）；Institute of Intelligent Manufacturing, Nanjing Tech University（南京工业大学智能制造研究院）；School of Computer Science, Shaanxi Normal University（陕西师范大学计算机学院）；College of Computer Science, Northeastern University（东北大学计算机科学与工程学院）；Faculty of Technology, Bielefeld University（比勒费尔德大学技术学院）
