通过平衡模态贡献实现鲁棒多模态语义分割
Robust Multimodal Semantic Segmentation with Balanced Modality Contributions
AI总结：
针对多模态分割中主导模态退化导致性能下降的问题，本文提出 EQUISeg 框架，通过平等模态编码、CMTB 融合与 SGM 相互引导平衡各模态贡献，提升退化条件下的鲁棒性。
摘要：
多模态语义分割通过利用跨模态互补性来增强模型的鲁棒性。然而，现有方法往往受到模态依赖不平衡的困扰：在真实场景中，一旦主导模态发生退化，整体性能就会显著下降。因此，模态平衡已成为实用多模态分割面临的一项关键挑战。为解决这一问题，我们提出了 EQUISeg，这是一个通过对各模态进行平等编码来平衡模态贡献的多模态分割框架。EQUISeg 构建于四阶段跨模态 Transformer 块（Cross-modal Transformer Block，CMTB）之上，可实现高效的多模态融合与分层选择。此外，我们设计了自引导模块（Self-guided Module，SGM），该模块通过引入相互引导机制来缓解模态不平衡，使每种模态都能自适应调整其贡献，并在退化条件下增强鲁棒性。在多个数据集上进行的大量实验表明，EQUISeg 取得了显著的性能提升，并有效缓解了分割任务中模态不平衡带来的不利影响。
英文摘要：
Multimodal semantic segmentation enhances model robustness by exploiting cross-modal complementarities. However, existing methods often suffer from imbalanced modal dependencies, where overall performance degrades significantly once a dominant modality deteriorates in real-world scenarios. Thus, modality balance has become acritical challenge for practical multimodal segmentation. To address this issue, we propose EQUISeg, a multimodal segmentation framework that balances modality contributions through equal encoding of modalities. Built upon a four-stage Cross-modal Transformer Block(CMTB), EQUISeg enables efficient multimodal fusion and hierarchical selection. Furthermore, we design a Self-guided Module(SGM) that mitigates modality imbalance by introducing a mutual guidance mechanism, enabling each modality to adaptively adjust its contribution and enhance robustness under degraded conditions. Extensive experiments on multiple datasets demonstrate that EQUISeg achieves significant performance gains and effectively alleviates the adverse effects of modality imbalance in segmentation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
