BigNAS：使用大型单阶段模型扩展神经架构搜索
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models
摘要：
神经结构搜索（NAS）已经显示出有希望的结果，发现既准确又快速的模型。对于NAS，训练一个一次性模型已经成为一种流行的策略，可以使用一组共享权重对不同架构（子模型）的相对质量进行排名。然而，虽然单次模型权重可以有效地对不同的网络架构进行排名，但这些共享权重的绝对精度通常远低于独立训练获得的精度。为了进行补偿，现有方法假设权重必须在搜索完成后重新训练、微调或以其他方式进行后处理。这些步骤显著增加了计算需求以及架构搜索和模型部署的复杂性。在这项工作中，我们提出了BigNAS，一种挑战传统智慧的方法，即权重的后处理是获得良好的预测精度所必需的。在没有额外的再训练或后处理步骤的情况下，我们能够在ImageNet上训练一组共享权重，并使用这些权重获得大小从200到1000 MFLOP的子模型。我们发现的模型家族BigNASModels实现了76.5%至80.9%的前1精度，超过了该范围内的最先进模型，包括EfficientNets和Once-for-All网络，无需额外的再训练或后处理。我们提出了烧蚀研究和分析，以进一步了解所提出的BigNASModels。
英文摘要：
Neural architecture search (NAS) has shown promising results discovering models that are both accurate and fast. For NAS, training a one-shot model has become a popular strategy to rank the relative quality of different architectures (child models) using a single set of shared weights. However, while one-shot model weights can effectively rank different network architectures, the absolute accuracies from these shared weights are typically far below those obtained from stand-alone training. To compensate, existing methods assume that the weights must be retrained, finetuned, or otherwise post-processed after the search is completed. These steps significantly increase the compute requirements and complexity of the architecture search and model deployment. In this work, we propose BigNAS, an approach that challenges the conventional wisdom that post-processing of the weights is necessary to get good prediction accuracies. Without extra retraining or post-processing steps, we are able to train a single set of shared weights on ImageNet and use these weights to obtain child models whose sizes range from 200 to 1000 MFLOPs. Our discovered model family, BigNASModels, achieve top-1 accuracies ranging from 76.5% to 80.9%, surpassing state-of-the-art models in this range including EfficientNets and Once-for-All networks without extra retraining or post-processing. We present ablative study and analysis to further understand the proposed BigNASModels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
