CoopSubNet：有限训练预算下深度网络数据驱动正则化的协作子网络
CoopSubNet: Cooperating Subnetwork for Data-Driven Regularization of Deep Networks under Limited Training Budgets
摘要：
深度网络是当前机器学习范式的一个组成部分。它们固有的学习数据和各种目标变量之间复杂函数映射的能力，同时发现隐藏的任务驱动功能，使其成为各种应用程序中的强大技术。尽管如此，这些网络的成功通常依赖于足够的训练数据的可用性，以优化大量的自由参数，同时避免过拟合，特别是对于具有大容量的网络。在培训预算有限的情况下，例如，针对具有有限标记样本的监督任务，已经应用了几种通用和/或特定于任务的正则化技术，包括数据增强，以提高深度网络的泛化能力。通常，这种正则化是独立于该数据或训练场景引入的，因此必须进行调整、测试和修改以满足特定网络的需求。在本文中，我们提出了一种新的正则化框架，该框架由待学习的特征空间的种群级统计数据驱动。正则化的形式是\textbf{cooperating subnetwork}，它是一个自动编码器架构，连接到特征空间，并结合主网络进行训练。我们介绍了架构和训练方法，并证明了所提出的基于合作网络的正则化在各种任务和架构的文献的有效性。我们的代码可在\url {https：//www.example.com}上免费获得github.com/riddhishb/CoopSubNet
英文摘要：
Deep networks are an integral part of the current machine learning paradigm. Their inherent ability to learn complex functional mappings between data and various target variables, while discovering hidden, task-driven features, makes them a powerful technology in a wide variety of applications. Nonetheless, the success of these networks typically relies on the availability of sufficient training data to optimize a large number of free parameters while avoiding overfitting, especially for networks with large capacity. In scenarios with limited training budgets, e.g., supervised tasks with limited labeled samples, several generic and/or task-specific regularization techniques, including data augmentation, have been applied to improve the generalization of deep networks.Typically such regularizations are introduced independently of that data or training scenario, and must therefore be tuned, tested, and modified to meet the needs of a particular network. In this paper, we propose a novel regularization framework that is driven by the population-level statistics of the feature space to be learned. The regularization is in the form of a \textbf{cooperating subnetwork}, which is an auto-encoder architecture attached to the feature space and trained in conjunction with the primary network. We introduce the architecture and training methodology and demonstrate the effectiveness of the proposed cooperative network-based regularization in a variety of tasks and architectures from the literature. Our code is freely available at \url{https://github.com/riddhishb/CoopSubNet
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
