arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.07651cs.CVcs.AIcs.LG
收藏

上下文归一化层及其应用

Context Normalization Layer with Applications

Sorbonne Paris Nord University(巴黎北大学);University of Versailles(凡尔赛大学);Université Paris-Saclay(巴黎萨克雷大学);Center for Development of Advanced Technologies(先进技术发展中心)

Bilal Faye, Mohamed-Djallel Dilmi, Hanane Azzag, Mustapha Lebbah, Djamel Bouchaffra

AI总结:

针对批归一化依赖批量大小且忽略数据分组的问题,本文提出面向图像的上下文归一化,根据样本特征自适应缩放,提升收敛速度与性能,并在多数据集上验证优于标准归一化方法。

摘要:

归一化是一种将数据转换为更可用表示的预处理步骤。作为深度神经网络(DNNs)的一部分,批归一化(BN)技术利用归一化来解决内部协变量偏移问题。它可以被封装为通用模块,并已被广泛集成到各种 DNN 中,以稳定和加速训练,据信还能带来更好的泛化能力。然而,BN 的效果依赖于小批量大小,并且在估计总体统计量时没有考虑数据集中可能存在的任何分组或簇。本研究针对图像数据提出了一种称为上下文归一化的新归一化技术。该方法根据每个样本的特征调整特征缩放,通过使数据值适应目标任务的上下文,提高模型的收敛速度和性能。上下文归一化的有效性在多个数据集上得到验证,并将其性能与其他标准归一化技术进行了比较。

英文摘要:

Normalization is a pre-processing step that converts the data into a more usable representation. As part of the deep neural networks (DNNs), the batch normalization (BN) technique uses normalization to address the problem of internal covariate shift. It can be packaged as general modules, which have been extensively integrated into various DNNs, to stabilize and accelerate training, presumably leading to improved generalization. However, the effect of BN is dependent on the mini-batch size and it does not take into account any groups or clusters that may exist in the dataset when estimating population statistics. This study proposes a new normalization technique, called context normalization, for image data. This approach adjusts the scaling of features based on the characteristics of each sample, which improves the model's convergence speed and performance by adapting the data values to the context of the target task. The effectiveness of context normalization is demonstrated on various datasets, and its performance is compared to other standard normalization techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑