DCLP：带有课程对比学习的神经架构预测器
DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning
AI总结：
针对NAS中神经预测器训练数据成本高、无监督方法易性能崩溃的问题，提出课程引导对比学习框架DCLP，通过难度排序的课程调度提升训练稳定性与效率，性能优于现有预测器。
摘要：
神经预测器在神经架构搜索（NAS）的评估过程中展现出巨大潜力。然而，当前基于预测器的方法忽略了一个事实：训练预测器需要大量已训练的神经网络作为有标签训练集，而获取这些数据的成本很高。因此，利用预测器进行NAS的关键问题在于，如何用尽可能少的已训练神经网络训练出高性能的预测器。尽管一些方法尝试通过无监督学习解决该问题，但它们往往会导致预测不准确。我们认为，面向通用图数据的无监督任务对神经网络来说难度过高，会导致NAS中的无监督训练容易出现性能崩溃。为解决这一问题，我们提出了一种面向神经预测器的课程引导对比学习框架（DCLP）。我们的方法通过设计一种新颖的课程来简化对比任务，以增强对比训练期间无标签训练数据分布的稳定性。具体而言，我们提出了一个调度器，它根据每个数据的对比难度对训练数据进行排序，再按顺序将其输入对比学习器。该方法集中了训练数据分布，使对比训练更高效。通过使用我们的方法，对比学习器能够在平滑的学习曲线上通过无监督数据逐步学习特征表示，避免了训练数据分布变化过大可能导致的性能崩溃。我们通过实验证明，与现有预测器相比，DCLP具有更高的准确率和效率，并且在与搜索策略结合时，在各种搜索空间中展现出发现更优架构的可观潜力。我们的代码可在以下网址获取：https://github.com/Zhengsh123/DCLP。
英文摘要：
Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a Curricumum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: https://github.com/Zhengsh123/DCLP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
