RegBN：具有正则化的多模态数据批归一化
RegBN: Batch Normalization of Multimodal Data with Regularization
AI总结：
本文提出结合正则化的多模态数据批归一化方法RegBN，利用Frobenius范数消除混杂因素与依赖性副作用，无需可学习参数，在多领域多架构上验证了其有效性和广泛适用性。
摘要：
近年来，神经网络在多模态数据整合方面的显著成功，推动了人们对整合多源传感器捕获的高维数据的兴趣激增。然而，异构多模态数据的整合提出了重大挑战，因为此类异构数据源之间的混杂效应和依赖性会引入不需要的变异性和偏差，导致多模态模型的性能次优。因此，在融合之前对从数据模态中提取的低级或高级特征进行归一化变得至关重要。本文提出了一种新颖的多模态数据归一化方法，称为 RegBN，它结合了正则化。RegBN 使用 Frobenius 范数作为正则化项，以解决混杂因素和不同数据源之间潜在依赖性的副作用。所提出的方法在多种模态上具有良好的泛化能力，并且消除了对可学习参数的需求，从而简化了训练和推理。我们在来自五个研究领域的八个数据库上验证了 RegBN 的有效性，这些数据库涵盖了语言、音频、图像、视频、深度、表格和 3D MRI 等多种模态。所提出的方法展示了在多层感知机、卷积神经网络和视觉 Transformer 等不同架构上的广泛适用性，能够对多模态神经网络中的低级和高级特征进行有效归一化。RegBN 可在 \url{https://github.com/mogvision/regbn} 获取。
英文摘要：
Recent years have witnessed a surge of interest in integrating high-dimensional data captured by multisource sensors, driven by the impressive success of neural networks in the integration of multimodal data. However, the integration of heterogeneous multimodal data poses a significant challenge, as confounding effects and dependencies among such heterogeneous data sources introduce unwanted variability and bias, leading to suboptimal performance of multimodal models. Therefore, it becomes crucial to normalize the low- or high-level features extracted from data modalities before their fusion takes place. This paper introduces a novel approach for the normalization of multimodal data, called RegBN, that incorporates regularization. RegBN uses the Frobenius norm as a regularizer term to address the side effects of confounders and underlying dependencies among different data sources. The proposed method generalizes well across multiple modalities and eliminates the need for learnable parameters, simplifying training and inference. We validate the effectiveness of RegBN on eight databases from five research areas, encompassing diverse modalities such as language, audio, image, video, depth, tabular, and 3D MRI. The proposed method demonstrates broad applicability across different architectures such as multilayer perceptrons, convolutional neural networks, and vision transformers, enabling effective normalization of both low- and high-level features in multimodal neural networks. RegBN is available at \url{https://github.com/mogvision/regbn}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
