基于局部自监督学习的神经结构搜索控制器预训练
Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning
摘要：
神经结构搜索（NAS）促进了机器学习的各个领域。尽管它的突出贡献，许多人批评了高计算成本的内在局限性。我们的目标是改善这一点，提出了一个预训练计划，可以普遍适用于基于网络的NAS。我们的方法，基于局部的自监督分类任务，利用网络架构的结构相似性，以获得良好的架构表示。我们将我们的方法纳入神经结构优化（NAO）中，以分析预训练的嵌入及其有效性，并强调增加度量学习损失对NAS带来了有利的影响。我们的代码可在\url{https://github.com/Multi-Objective-NAS/self-supervised-nas}上获得。
英文摘要：
Neural architecture search (NAS) has fostered various fields of machine learning. Despite its prominent dedications, many have criticized the intrinsic limitations of high computational cost. We aim to ameliorate this by proposing a pretraining scheme that can be generally applied to controller-based NAS. Our method, locality-based self-supervised classification task, leverages the structural similarity of network architectures to obtain good architecture representations. We incorporate our method into neural architecture optimization (NAO) to analyze the pretrained embeddings and its effectiveness and highlight that adding metric learning loss brings a favorable impact on NAS. Our code is available at \url{https://github.com/Multi-Objective-NAS/self-supervised-nas}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
