PRIMED: 通过偏见竞争实现适应性模态抑制的指引用视听分割
PRIMED: Adaptive Modality Suppression for Referring Audio-Visual Segmentation via Biased Competition
AI总结：
PRIMED通过偏见竞争理论，实现适应性模态抑制，提升指引用视听分割的准确性，通过模态先验解码器和token蒸馏器增强多模态融合，实验表明其在Ref-AVS基准上达到最优性能。
摘要：
指引用视听分割（Ref-AVS）旨在基于视觉、听觉和文本指引用线索定位和分割目标对象。该任务具有挑战性，因为不同模态的相关性在指引用表达和场景中变化，而现有方法通常将多模态线索视为同质输入进行融合、提示或推理，导致对无关或误导性模态敏感。为解决此问题，我们提出PRIMED，受认知神经科学中的偏见竞争理论启发，显式建模视觉感知和语言驱动的先验调节，并通过适应性模态抑制实现更精确的Ref-AVS。具体而言，模态先验解码器首先估计指引用是否主要依赖音频、视觉或其联合交互，生成模态先验以适应性引导高层注意力。Token蒸馏器进一步从高层特征中提取紧凑的全局视觉token，并在竞争感知的跨模态融合模块中共享，以提供层次化的全局上下文。此外，我们引入了空间感知的语义对齐损失，通过对比学习进一步增强前景-背景区分。在Ref-AVS基准上的广泛实验表明，PRIMED在整体性能上达到最优。
英文摘要：
Referring Audio-Visual Segmentation (Ref-AVS) seeks to localize and segment target objects in video frames based on visual, auditory, and textual referring cues. The task is challenging because the relevance of different modalities varies across referring expressions and scenes, while existing methods typically treat multimodal cues as homogeneous inputs for fusion, prompting, or reasoning, making them vulnerable to irrelevant or misleading modalities. To address this problem, we propose PRIMED, inspired by the biased competition theory in cognitive neuroscience, which explicitly models both visual perception and language-driven prior modulation, and enables more accurate Ref-AVS by adaptive modality suppression. Specifically, a Modality Prior Decoder first estimates whether the referring expression relies primarily on audio, vision, or their joint interaction, generating a modality prior to adaptively guide high-level attention. A Token Distiller further extracts compact global visual tokens from high-level features and shares them across Competition-aware Cross-modal Fusion modules to provide hierarchical global context. Additionally, we introduce a Spatial-Aware Semantic Alignment loss to further enhance foreground-background discrimination through contrastive learning. Extensive experiments on the Ref-AVS benchmark demonstrate that PRIMED achieves state-of-the-art overall performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
