DifferSeg: 通过差分感知与频率引导实现多样化的多模态二值分割
DifferSeg: Towards Diverse Multimodal Binary Segmentation via Differential Perception and Frequency Guidance
AI总结：
提出DifferSeg框架，通过差分感知融合模块自适应对齐多模态特征，并设计频率引导解码器平衡高低频表示，在29个公开数据集上超越67种方法。
摘要：
在许多二值分割任务中，大多数多模态方法依赖于固定的特征拼接进行跨模态交互，以及由低频语义主导的简单解码器设计。然而，它们忽略了两个关键挑战：一是缺乏处理模态差异和互补性的自适应机制，二是缺少平衡高低频表示的高效解码策略。在这项工作中，我们提出了一个简单而通用的多模态二值分割框架，称为DifferSeg，以同时解决这两个问题。借助差分感知融合（DPF）模块，DifferSeg使用可学习的差分算子自适应地对齐多模态特征，并通过残差融合增强其互补性，有效缓解模态不匹配和融合冗余。此外，我们设计了一个频率引导解码器（FGD），构建跨频率交互和多路径上采样，以保持细节高频结构与语义低频表示之间的一致性，确保细粒度边界恢复和噪声抑制。得益于这些设计，DifferSeg可以轻松泛化到各种二值分割任务，包括自然和医学模态。无需额外技巧，它在涉及18个下游任务的29个公开数据集上持续超越67种最先进方法，展示了卓越的泛化能力和分割精度。代码和预训练模型将在链接处提供。
英文摘要：
In many binary segmentation tasks, most multimodal methods rely on fixed feature concatenation for cross-modal interaction and straightforward decoder designs dominated by low-frequency semantics. However, they ignore two key challenges: one is the lack of an adaptive mechanism to handle modality discrepancies and complementarity, and the other is the absence of an efficient decoding strategy to balance both high- and low-frequency representations. In this work, we propose a simple yet general multimodal binary segmentation framework, termed DifferSeg, to address both problems simultaneously. With the help of the differential perception fusion (DPF) module, DifferSeg employs learnable differential operators to adaptively align multimodal features and enhance their complementarity through residual fusion, effectively mitigating modality mismatch and fusion redundancy. In addition, we design a frequency-guided decoder (FGD) that builds cross-frequency interactions and multi-path upsampling to maintain consistency between detailed high-frequency structures and semantic low-frequency representations, ensuring fine-grained boundary recovery and noise suppression. % Benefiting from these designs, DifferSeg can be easily generalized to diverse binary segmentation tasks, including both natural and medical modalities. Without bells and whistles, it consistently surpasses 67 state-of-the-art methods across 29 public datasets involving 18 downstream tasks, demonstrating superior generalization and segmentation accuracy.Code and pretrained models will be available at the Link.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
