核归一化卷积网络
Kernel Normalized Convolutional Networks
AI总结：
针对BatchNorm小批量性能差且不适用差分隐私的问题，提出KernelNorm及KNConvNets，在分类和分割任务中兼具批量无关性与优异性能。
摘要：
现有的卷积神经网络架构经常依赖批量归一化（BatchNorm）来有效训练模型。然而，BatchNorm在小批量规模下表现不佳，并且不适用于差分隐私。为解决这些局限，我们提出核归一化（KernelNorm）和核归一化卷积层，并将其纳入核归一化卷积网络（KNConvNets）作为主要构建模块。我们实现了与最先进ResNets相对应的KNConvNets，同时放弃BatchNorm层。通过大量实验，我们表明KNConvNets在图像分类和语义分割中相比BatchNorm对应模型取得更高或具有竞争力的性能。在非私有和差分私有训练中，它们还显著优于包括基于层归一化和组归一化在内的不依赖批量的竞争方法。鉴于此，KernelNorm将层归一化和组归一化的批量无关特性与BatchNorm的性能优势结合起来。
英文摘要：
Existing convolutional neural network architectures frequently rely upon batch normalization (BatchNorm) to effectively train the model. BatchNorm, however, performs poorly with small batch sizes, and is inapplicable to differential privacy. To address these limitations, we propose the kernel normalization (KernelNorm) and kernel normalized convolutional layers, and incorporate them into kernel normalized convolutional networks (KNConvNets) as the main building blocks. We implement KNConvNets corresponding to the state-of-the-art ResNets while forgoing the BatchNorm layers. Through extensive experiments, we illustrate that KNConvNets achieve higher or competitive performance compared to the BatchNorm counterparts in image classification and semantic segmentation. They also significantly outperform their batch-independent competitors including those based on layer and group normalization in non-private and differentially private training. Given that, KernelNorm combines the batch-independence property of layer and group normalization with the performance advantage of BatchNorm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
