arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.05274cs.CV
收藏

归一化卷积神经网络

Normalized Convolutional Neural Network

JLK Group(Research)(JLK研究集团)

Dongsuk Kim, Geonhee Lee, Myungjae Lee, Shin Uk Kang, Dongmin Kim

AI总结:

提出归一化卷积神经网络层,通过在卷积中归一化 im2col 矩阵行,实现自适应切片输入并提升微批次训练稳定性,具有普适性。

摘要:

我们引入了归一化卷积神经网络层,这是一种卷积网络中归一化的新方法。与传统方法不同,该层在卷积过程中对 im2col 矩阵的行进行归一化,使其天然适应切片输入,并与卷积核结构更好对齐。这种独特性使其有别于标准归一化技术,且无法直接集成到针对传统卷积操作优化的现有深度学习框架中。该方法具有普适性,适用于任何涉及卷积层的深度学习任务。通过在卷积过程内部进行归一化,它作为 Self-Normalizing Networks 的卷积化改造,在无需额外归一化层的情况下保持了其核心原则。值得注意的是,在微批次训练场景中,它始终优于其他独立于批次的归一化方法。这种性能提升源于对 im2col 矩阵行的标准化,理论上可带来更平滑的损失梯度和更高的训练稳定性。

英文摘要:

We introduce a Normalized Convolutional Neural Layer, a novel approach to normalization in convolutional networks. Unlike conventional methods, this layer normalizes the rows of the im2col matrix during convolution, making it inherently adaptive to sliced inputs and better aligned with kernel structures. This distinctive approach differentiates it from standard normalization techniques and prevents direct integration into existing deep learning frameworks optimized for traditional convolution operations. Our method has a universal property, making it applicable to any deep learning task involving convolutional layers. By inherently normalizing within the convolution process, it serves as a convolutional adaptation of Self-Normalizing Networks, maintaining their core principles without requiring additional normalization layers. Notably, in micro-batch training scenarios, it consistently outperforms other batch-independent normalization methods. This performance boost arises from standardizing the rows of the im2col matrix, which theoretically leads to a smoother loss gradient and improved training stability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑