通过多模态交互与池化注意力优化 RGB-D 语义分割
Optimizing rgb-d semantic segmentation through multi-modal interaction and pooling attention
AI总结：
针对室内 RGB-D 语义分割中 RGB 与深度信息利用不足的问题，提出 MIPANet，通过 MIM 深层交互融合和 PAM 多阶段注意力增强，在 NYUDv2 与 SUN-RGBD 上取得优于现有方法的性能。
摘要：
RGB-D 图像的语义分割涉及理解场景中物体的外观与空间关系，这需要谨慎考虑多种因素。然而，在室内环境中，简单输入 RGB 和深度图像往往只能获取相对有限的语义与空间信息，从而导致分割结果不够理想。为解决这一问题，我们提出了多模态交互与池化注意力网络（Multi-modal Interaction and Pooling Attention Network，MIPANet），这是一种旨在利用 RGB 与深度模态之间交互协同作用、优化互补信息利用的新方法。具体而言，我们在网络最深层加入多模态交互融合模块（Multi-modal Interaction Fusion Module，MIM）。该模块经过设计，用于促进 RGB 与深度信息的融合，使二者能够相互增强和校正。此外，我们在编码器的多个阶段引入池化注意力模块（Pooling Attention Module，PAM）。该模块用于放大网络提取的特征，并将模块输出有针对性地整合到解码器中，从而显著提升语义分割性能。实验结果表明，MIPANet 在 NYUDv2 和 SUN-RGBD 两个室内场景数据集上优于现有方法，突显了其在增强 RGB-D 语义分割方面的有效性。
英文摘要：
Semantic segmentation of RGB-D images involves understanding the appearance and spatial relationships of objects within a scene, which requires careful consideration of various factors. However, in indoor environments, the simple input of RGB and depth images often results in a relatively limited acquisition of semantic and spatial information, leading to suboptimal segmentation outcomes. To address this, we propose the Multi-modal Interaction and Pooling Attention Network (MIPANet), a novel approach designed to harness the interactive synergy between RGB and depth modalities, optimizing the utilization of complementary information. Specifically, we incorporate a Multi-modal Interaction Fusion Module (MIM) into the deepest layers of the network. This module is engineered to facilitate the fusion of RGB and depth information, allowing for mutual enhancement and correction. Additionally, we introduce a Pooling Attention Module (PAM) at various stages of the encoder. This module serves to amplify the features extracted by the network and integrates the module's output into the decoder in a targeted manner, significantly improving semantic segmentation performance. Our experimental results demonstrate that MIPANet outperforms existing methods on two indoor scene datasets, NYUDv2 and SUN-RGBD, underscoring its effectiveness in enhancing RGB-D semantic segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
