arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.00396cs.CV

Saliency-R1: 通过置信度引导的强化学习激励多模态大语言模型统一的显著性推理能力

Saliency-R1: Incentivizing Unified Saliency Reasoning Capability in MLLM with Confidence-Guided Reinforcement Learning

Northwestern Polytechnical University(西北工业大学);University of Science and Technology of China(中国科学技术大学)

Long Li, Shuichen Ji, Ziyang Luo, Zhihui Li, Dingwen Zhang, Junwei Han, Nian Liu

AI总结:

Saliency-R1通过置信度引导的强化学习,提升多模态大语言模型在显著性推理任务中的统一能力,实现显著物体检测、显著实例分割和共显著物体检测的高效处理。

摘要:

尽管多模态大语言模型(MLLMs)在高级视觉-语言推理方面表现出色,但它们缺乏对视觉显著性的内在意识,难以识别关键视觉元素。为弥合这一差距,我们提出了Saliency-R1,这是首个统一的MLLM框架,共同解决三个代表性且异质的显著性任务:显著物体检测(SOD)、显著实例分割(SIS)和共显著物体检测(CoSOD),增强模型的显著性推理能力。我们引入了一个带有结构化标签(<rg>、<ins>)的文本接口,用于编码区域级和实例级的指代表达,使一个指代分割器能够生成任务合适的掩码。为了高效训练MLLM,我们提出了一种新的单样本强化学习算法,即置信度引导的策略优化(CGPO)。CGPO在GRPO基础上,用基于奖励-置信度差异的单样本信号替代组归一化优势,从而减少计算浪费,缓解信号稀释,并降低训练开销。我们的模型在所有三个任务中均超过或匹配了稳健的开源/闭源MLLM和专用最先进的方法的性能,证明了我们框架在显著性推理中的有效性。

英文摘要:

Although multimodal large language models (MLLMs) excel in high-level vision-language reasoning, they lack inherent awareness of visual saliency, making it difficult to identify key visual elements. To bridge this gap, we propose Saliency-R1, the first unified MLLM framework that jointly tackles three representative and heterogeneous saliency tasks: Salient Object Detection (SOD), Salient Instance Segmentation (SIS), and Co-salient Object Detection (CoSOD), enhancing the model's capacity for saliency reasoning. We introduce a textual interface with structured tags (<rg>, <ins>) to encode region- and instance-level referring expressions, enabling a single referring segmenter to produce task-appropriate masks. To train the MLLM efficiently, we propose Confidence-Guided Policy Optimization (CGPO), a novel single-sample reinforcement learning algorithm. CGPO improves on GRPO by replacing group-normalized advantages with a per-sample signal based on reward-confidence discrepancy, thereby reducing computational waste, mitigating signal dilution, and lowering training overhead. Our model exceeds or matches the performance of robust open/closed-source MLLMs and specialized state-of-the-art methods across all three tasks, demonstrating the efficacy of our framework in saliency reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑