arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.15082cs.CV
收藏

单元变分信息瓶颈网络

Cell Variational Information Bottleneck Network

Zhonghua Zhai, Chen Ju, Jinsong Lan, Shuai Xiao

AI总结:

本文提出单元变分信息瓶颈网络cellVIB,通过堆叠VIB单元逐步增强正则化,在MNIST、CIFAR-10、PACS及人脸识别任务中验证了其对噪声的鲁棒性和泛化性能提升。

摘要:

在这项工作中,我们提出了单元变分信息瓶颈网络(cellVIB),这是一种使用信息瓶颈机制的卷积神经网络,能够以端到端训练方式与最新的前馈网络架构相结合。我们的单元变分信息瓶颈网络通过堆叠VIB单元构建,这些单元生成带有不确定性的特征图。随着层数加深,正则化效果会逐渐增强,而不是像Deep VIB那样直接对模型输出层施加过多的正则约束。在每个VIB单元下,前馈过程学习独立的均值项和标准差项,并基于它们预测高斯分布。反馈过程基于重参数化技巧以实现有效训练。本工作在MNIST数据集上进行了广泛分析,以验证每个VIB单元的有效性,并就VIB单元如何影响互信息提供了深入的分析。在CIFAR-10上进行的实验也证明,我们的cellVIB在训练期间对噪声标签以及在测试期间对损坏图像都具有鲁棒性。然后,我们在PACS数据集上验证了我们的方法,结果表明VIB单元能够显著提升基础模型的泛化性能。最后,在一个更复杂的表示学习任务——人脸识别中,我们的网络结构也取得了非常有竞争力的结果。

英文摘要:

In this work, we propose Cell Variational Information Bottleneck Network (cellVIB), a convolutional neural network using information bottleneck mechanism, which can be combined with the latest feedforward network architecture in an end-to-end training method. Our Cell Variational Information Bottleneck Network is constructed by stacking VIB cells, which generate feature maps with uncertainty. As layers going deeper, the regularization effect will gradually increase, instead of directly adding excessive regular constraints to the output layer of the model as in Deep VIB. Under each VIB cell, the feedforward process learns an independent mean term and an standard deviation term, and predicts the Gaussian distribution based on them. The feedback process is based on reparameterization trick for effective training. This work performs an extensive analysis on MNIST dataset to verify the effectiveness of each VIB cells, and provides an insightful analysis on how the VIB cells affect mutual information. Experiments conducted on CIFAR-10 also prove that our cellVIB is robust against noisy labels during training and against corrupted images during testing. Then, we validate our method on PACS dataset, whose results show that the VIB cells can significantly improve the generalization performance of the basic model. Finally, in a more complex representation learning task, face recognition, our network structure has also achieved very competitive results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑