一种用于无需批归一化的ResNet有效训练的残差块鲁棒初始化方法
A Robust Initialization of Residual Blocks for Effective ResNet Training without Batch Normalization
AI总结：
针对批归一化带来的实际问题，该研究通过微调残差块输出与跳跃连接的求和操作实现正确初始化，成功训练无归一化的类ResNet网络，在多个主流图像数据集上取得了有竞争力的结果。
摘要：
批归一化（Batch Normalization）是所有当前最优神经网络架构的核心组件。但由于它会带来诸多实际问题，近期大量研究致力于设计无需归一化的架构。本文表明，权重初始化是训练类ResNet无归一化网络的关键。具体而言，我们对残差块输出与跳跃连接分支的求和操作做了小幅修改，从而使整个网络得到正确初始化。实验显示，这种改进后的架构在CIFAR-10、CIFAR-100和ImageNet上取得了具有竞争力的结果，且无需额外正则化或算法修改。
英文摘要：
Batch Normalization is an essential component of all state-of-the-art neural networks architectures. However, since it introduces many practical issues, much recent research has been devoted to designing normalization-free architectures. In this paper, we show that weights initialization is key to train ResNet-like normalization-free networks. In particular, we propose a slight modification to the summation operation of a block output to the skip-connection branch, so that the whole network is correctly initialized. We show that this modified architecture achieves competitive results on CIFAR-10, CIFAR-100 and ImageNet without further regularization nor algorithmic modifications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
