用于RGB-T语义分割的变分概率融合网络
Variational Probabilistic Fusion Network for RGB-T Semantic Segmentation
AI总结：
针对RGB-T语义分割中的模态噪声、类别不平衡与模态偏差问题，提出VPFNet，通过变分特征融合采样并结合光照、类别先验实现鲁棒分割，在MFNet和PST900上达到SOTA性能。
摘要：
RGB-T语义分割通过融合RGB图像和热红外图像的不同模态特征，已被广泛用于处理光照条件较差的困难场景。现有方法试图寻找用于分割的最优融合特征，从而导致对模态噪声、类别不平衡和模态偏差敏感。为克服这些问题，本文提出一种新颖的变分概率融合网络（Variational Probabilistic Fusion Network，VPFNet），该网络将融合特征视为随机变量，并通过对融合特征多个样本下的分割结果进行平均来获得鲁棒分割。VPFNet中融合特征的随机样本生成由一个基于变分注意力设计的新颖变分特征融合模块（Variational Feature Fusion Module，VFFM）实现。为进一步避免类别不平衡和模态偏差，我们采用加权交叉熵损失，并引入光照和类别的先验信息来控制所提出的VFFM。在MFNet和PST900数据集上的实验结果表明，所提出的VPFNet能够达到最先进的分割性能。
英文摘要：
RGB-T semantic segmentation has been widely adopted to handle hard scenes with poor lighting conditions by fusing different modality features of RGB and thermal images. Existing methods try to find an optimal fusion feature for segmentation, resulting in sensitivity to modality noise, class-imbalance, and modality bias. To overcome the problems, this paper proposes a novel Variational Probabilistic Fusion Network (VPFNet), which regards fusion features as random variables and obtains robust segmentation by averaging segmentation results under multiple samples of fusion features. The random samples generation of fusion features in VPFNet is realized by a novel Variational Feature Fusion Module (VFFM) designed based on variation attention. To further avoid class-imbalance and modality bias, we employ the weighted cross-entropy loss and introduce prior information of illumination and category to control the proposed VFFM. Experimental results on MFNet and PST900 datasets demonstrate that the proposed VPFNet can achieve state-of-the-art segmentation performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
