HTMNet：一种基于Transformer-Mamba瓶颈多模态融合的透明与反射物体深度补全混合网络
HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion
AI总结：
提出HTMNet混合模型，通过融合CNN、Transformer与Mamba架构的编码器、瓶颈和解码器模块，首次将Mamba应用于透明与反射物体深度补全，在多个公开数据集上达到SOTA性能。
摘要：
透明和反射物体对深度传感器提出了重大挑战，导致深度信息不完整，这会对下游的机器人感知与操作任务产生不利影响。为了解决这一问题，我们提出了HTMNet，一种融合Transformer、CNN和Mamba架构的新型混合模型。编码器基于双分支CNN-Transformer框架，瓶颈融合模块采用Transformer-Mamba架构，解码器则建立在多尺度融合模块之上。我们引入了一个基于自注意力机制和状态空间模型的新型多模态融合模块，这标志着Mamba架构首次应用于透明物体深度补全领域，并展现了其巨大的潜力。此外，我们为解码器设计了一个创新的多尺度融合模块，该模块结合了通道注意力、空间注意力和多尺度特征提取技术，通过下采样融合策略有效整合多尺度特征。在多个公开数据集上的广泛评估表明，我们的模型达到了当前最先进（SOTA）的性能，验证了该方法的有效性。
英文摘要：
Transparent and reflective objects pose significant challenges for depth sensors, resulting in incomplete depth information that adversely affects downstream robotic perception and manipulation tasks. To address this issue, we propose HTMNet, a novel hybrid model integrating Transformer, CNN, and Mamba architectures. The encoder is based on a dual-branch CNN-Transformer framework, the bottleneck fusion module adopts a Transformer-Mamba architecture, and the decoder is built upon a multi-scale fusion module. We introduce a novel multimodal fusion module grounded in self-attention mechanisms and state space models, marking the first application of the Mamba architecture in the field of transparent object depth completion and revealing its promising potential. Additionally, we design an innovative multi-scale fusion module for the decoder that combines channel attention, spatial attention, and multi-scale feature extraction techniques to effectively integrate multi-scale features through a down-fusion strategy. Extensive evaluations on multiple public datasets demonstrate that our model achieves state-of-the-art(SOTA) performance, validating the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
