基于多调和样条的大规模神经网络结构搜索
Large Scale Neural Architecture Search with Polyharmonic Splines
摘要:
神经架构搜索(NAS)是一个强大的工具,可以自动设计用于许多任务的深度神经网络,包括图像分类。由于搜索阶段的计算负担很大,到目前为止,大多数NAS方法都集中在小型平衡数据集上。大规模进行NAS的所有尝试都采用了小的代理集,然后通过复制或堆叠搜索的单元将学习到的架构转移到更大的数据集。我们提出了一种基于多调和样条的NAS方法,可以直接在大规模,不平衡的目标数据集上进行搜索。我们在ImageNet 22 K基准测试[16]上证明了我们方法的有效性,该测试包含1400万张图像,这些图像以高度不平衡的方式分布在21,841个类别中。通过直接在ImageNet 22 K上探索ResNet [23]和Big-Little Net ResNext [11]架构的搜索空间,我们的多调和样条NAS方法设计了一个模型,该模型在ImageNet 22 K上实现了40.03%的top-1准确度,比具有类似全局批量大小的现有技术绝对提高了3.13%[15]。
英文摘要:
Neural Architecture Search (NAS) is a powerful tool to automatically design deep neural networks for many tasks, including image classification. Due to the significant computational burden of the search phase, most NAS methods have focused so far on small, balanced datasets. All attempts at conducting NAS at large scale have employed small proxy sets, and then transferred the learned architectures to larger datasets by replicating or stacking the searched cells. We propose a NAS method based on polyharmonic splines that can perform search directly on large scale, imbalanced target datasets. We demonstrate the effectiveness of our method on the ImageNet22K benchmark[16], which contains 14 million images distributed in a highly imbalanced manner over 21,841 categories. By exploring the search space of the ResNet [23] and Big-Little Net ResNext [11] architectures directly on ImageNet22K, our polyharmonic splines NAS method designed a model which achieved a top-1 accuracy of 40.03% on ImageNet22K, an absolute improvement of 3.13% over the state of the art with similar global batch size [15].
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。