arXiv 2511.130472025-11-17cs.CVcs.RO

DiffPixelFormer：用于RGB-D室内场景分割的差分像素感知Transformer

DiffPixelFormer: Differential Pixel-Aware Transformer for RGB-D Indoor Scene Segmentation

State Key Laboratory of Robotics and System, Harbin Institute of Technology（机器人系统国家重点实验室，哈尔滨工业大学）；Institute for Infocomm Research, A*STAR（信息通信研究机构，A*STAR）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）

Yan Gong, Jianli Lu, Yongsheng Gao, Jie Zhao, Xiaojuan Zhang, Susanto Rahardja

备注：: 11 pages, 5 figures, 5 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有RGB-D室内语义分割方法计算成本高、跨模态特征对齐不足的问题，本文提出DiffPixelFormer模型，通过IIMIB模块与动态融合策略实现精准跨模态对齐，在两个基准数据集上取得了优于对比方法的分割精度。

摘要：

室内语义分割是计算机视觉与机器人领域的基础技术，可为自主导航、增强现实、智能环境等应用提供支撑。尽管RGB-D融合可利用互补的外观与几何线索，但现有方法往往依赖计算开销极大的交叉注意力机制，且对模态内与模态间的特征关系建模不足，进而导致特征对齐不精准、特征表示判别性有限。为解决上述问题，本文提出DiffPixelFormer，一款面向RGB-D室内场景分割的差分像素感知Transformer，可同时增强模态内表示并对模态间交互进行建模。其核心组件为模态内-模态间交互块（IIMIB），该模块通过自注意力捕获模态内长程依赖关系，并借助差分共享模态间（DSIM）模块对模态间交互建模，以解耦模态特有线索与共享线索，实现细粒度像素级跨模态对齐。此外，本文提出的动态融合策略可根据场景特性平衡各模态贡献，充分挖掘RGB-D信息。在SUN RGB-D与NYUDv2基准上开展的大量实验表明，DiffPixelFormer-L的mIoU得分分别达到54.28%与59.95%，较DFormer-L分别提升1.78%与2.75%。代码可在https://github.com/gongyan1/DiffPixelFormer获取。

英文摘要：

Indoor semantic segmentation is fundamental to computer vision and robotics, supporting applications such as autonomous navigation, augmented reality, and smart environments. Although RGB-D fusion leverages complementary appearance and geometric cues, existing methods often depend on computationally intensive cross-attention mechanisms and insufficiently model intra- and inter-modal feature relationships, resulting in imprecise feature alignment and limited discriminative representation. To address these challenges, we propose DiffPixelFormer, a differential pixel-aware Transformer for RGB-D indoor scene segmentation that simultaneously enhances intra-modal representations and models inter-modal interactions. At its core, the Intra-Inter Modal Interaction Block (IIMIB) captures intra-modal long-range dependencies via self-attention and models inter-modal interactions with the Differential-Shared Inter-Modal (DSIM) module to disentangle modality-specific and shared cues, enabling fine-grained, pixel-level cross-modal alignment. Furthermore, a dynamic fusion strategy balances modality contributions and fully exploits RGB-D information according to scene characteristics. Extensive experiments on the SUN RGB-D and NYUDv2 benchmarks demonstrate that DiffPixelFormer-L achieves mIoU scores of 54.28% and 59.95%, outperforming DFormer-L by 1.78% and 2.75%, respectively. Code is available at https://github.com/gongyan1/DiffPixelFormer.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。