重新思考早期融合策略以改进多模态图像分割
Rethinking Early-Fusion Strategies for Improved Multimodal Image Segmentation
AI总结：
针对现有RGB-T多模态分割方法参数和计算开销大的问题，提出基于早期融合与特征聚类的EFNet网络及轻量级多尺度解码器，以更低成本超越现有最优方法。
摘要：
RGB和热图像融合在低光照条件下具有展现改进语义分割的巨大潜力。现有方法通常采用双分支编码器框架进行多模态特征提取，并设计复杂的特征融合策略以实现多模态语义分割的特征提取与融合。然而，这些方法在特征提取和融合过程中需要大量的参数更新和计算开销。为解决这一问题，我们提出了一种基于早期融合策略和简单而有效的特征聚类的新型多模态融合网络（EFNet），用于训练高效的RGB-T语义分割。此外，我们还提出了一种基于欧氏距离的轻量级且高效的多尺度特征聚合解码器。我们在不同数据集上验证了该方法的有效性，并以更少的参数和计算量超越了先前的最先进方法。
英文摘要：
RGB and thermal image fusion have great potential to exhibit improved semantic segmentation in low-illumination conditions. Existing methods typically employ a two-branch encoder framework for multimodal feature extraction and design complicated feature fusion strategies to achieve feature extraction and fusion for multimodal semantic segmentation. However, these methods require massive parameter updates and computational effort during the feature extraction and fusion. To address this issue, we propose a novel multimodal fusion network (EFNet) based on an early fusion strategy and a simple but effective feature clustering for training efficient RGB-T semantic segmentation. In addition, we also propose a lightweight and efficient multi-scale feature aggregation decoder based on Euclidean distance. We validate the effectiveness of our method on different datasets and outperform previous state-of-the-art methods with lower parameters and computation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
