通过网络编码学习通用神经结构
Learning Versatile Neural Architectures by Propagating Network Codes
摘要：
这项工作探讨了如何设计一个单一的神经网络能够适应多个异构的视觉任务，如图像分割，3D检测和视频识别。这个目标是具有挑战性的，因为网络架构搜索（NAS）的空间和方法在不同的任务是不一致的。我们从两个方面解决这个挑战。我们首先为多个任务引入了一个统一的设计空间，并在许多广泛使用的数据集上构建了一个多任务NAS基准测试（NAS-Bench-MR），包括ImageNet，Cityscapes，KITTI和HMDB 51。我们进一步提出了网络编码传播（NCP），它反向传播神经预测器的梯度，以沿着所需的梯度方向直接更新架构代码，以解决各种任务。通过这种方式，NCP可以在几秒钟内在我们的大搜索空间中找到最佳架构配置。与通常专注于单个任务的NAS的现有技术不同，NCP具有几个独特的优点。(1)NCP将架构优化从数据驱动转变为架构驱动，实现了不同数据分布的多任务间的架构联合搜索。(2)NCP从网络代码而不是原始数据中学习，使其能够跨数据集有效地更新架构。(3)除了我们的NAS-Bench-MR，NCP在其他NAS基准测试中表现良好，例如NAS-Bench-201。(4)对任务间、任务间和任务内NCP的深入研究突出了跨任务神经结构设计的重要性，即，多任务神经结构和不同任务之间的结构转换。代码可在https://github.com/dingmyu/NCP上获得。
英文摘要：
This work explores how to design a single neural network capable of adapting to multiple heterogeneous vision tasks, such as image segmentation, 3D detection, and video recognition. This goal is challenging because both network architecture search (NAS) spaces and methods in different tasks are inconsistent. We solve this challenge from both sides. We first introduce a unified design space for multiple tasks and build a multitask NAS benchmark (NAS-Bench-MR) on many widely used datasets, including ImageNet, Cityscapes, KITTI, and HMDB51. We further propose Network Coding Propagation (NCP), which back-propagates gradients of neural predictors to directly update architecture codes along the desired gradient directions to solve various tasks. In this way, optimal architecture configurations can be found by NCP in our large search space in seconds. Unlike prior arts of NAS that typically focus on a single task, NCP has several unique benefits. (1) NCP transforms architecture optimization from data-driven to architecture-driven, enabling joint search an architecture among multitasks with different data distributions. (2) NCP learns from network codes but not original data, enabling it to update the architecture efficiently across datasets. (3) In addition to our NAS-Bench-MR, NCP performs well on other NAS benchmarks, such as NAS-Bench-201. (4) Thorough studies of NCP on inter-, cross-, and intra-tasks highlight the importance of cross-task neural architecture design, i.e., multitask neural architectures and architecture transferring between different tasks. Code is available at https://github.com/dingmyu/NCP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
