DTFormer：用于RGB-D分割的文本引导语义对齐
DTFormer: Text-Guided Semantic Alignment for RGB-D Segmentation
浏览论文内容
中文总结 AI 辅助
DTFormer提出一种RGB-D-文本三模态语义分割框架，通过文本引导语义对齐模块将语言先验编码为语义原型并与多模态特征对齐，在多个基准上持续提升分割性能，兼具简洁高效。
中文摘要 AI 辅助
RGB-D语义分割通过融合RGB和深度信息取得了显著进展，但主流模型仍几乎完全依赖像素级监督来学习特征，缺乏直接的高层语义约束。这引出了一个核心问题——诸如语言先验之类的外部知识能否为主流RGB-D分割模型注入更强的语义判别性。我们提出了DTFormer，一种新颖的三模态（RGB-D-文本）语义分割框架。其核心是文本引导语义对齐模块（TSAM），该模块首先将文本线索编码为一组语义原型，然后在多个编码器和解码器层将多模态RGB-D特征与这些原型显式对齐。这种设计对表示学习施加了强语义正则化，引导网络朝向更具判别性的特征。在多个基准上的大量实验表明，DTFormer在保持简洁高效的同时取得了持续的性能提升。我们的结果表明，显式语义对齐为改进RGB-D语义分割提供了一条有效且实用的途径。代码将在论文被接收后发布。
英文摘要
RGB-D semantic segmentation has made notable progress by fusing RGB and Depth, yet mainstream models still learn features almost exclusively from pixel-level supervision, lacking direct high-level semantic constraints. This raises a central question-can external knowledge such as language priors inject stronger semantic discriminability into mainstream RGB-D segmentation models. We present DTFormer, a novel tri-modal (RGB-D-Text) semantic segmentation framework. At its core is Text-guided Semantic Alignment Module (TSAM) that first encodes textual cues into a set of semantic prototypes and then explicitly aligns multi-modal RGB-D features with these prototypes at multiple encoder and decoder layers. This design imposes strong semantic regularization on representation learning, guiding the network toward more discriminative features. Extensive experiments on multiple benchmarks show that DTFormer delivers consistent gains while remaining simple and efficient. Our results demonstrate that explicit semantic alignment offers an effective and practical route to improving RGB-D semantic segmentation. The code will be released upon acceptance.
发表机构
- Southeast University（东南大学）
- Technical University of Munich（慕尼黑工业大学）
- City University of Macau（澳门城市大学）
- University of Science and Technology of China（中国科学技术大学）
机构由 AI 辅助整理，请以论文原文为准。
