面向多任务视觉定位的渐进式语言引导视觉学习
Progressive Language-guided Visual Learning for Multi-Task Visual Grounding
AI总结：
本文提出PLVL框架，通过逐步注入语言信息到视觉主干网络以省去额外跨模态融合模块，并设计多任务头利用REC定位中心辅助RES分割，实现协同预测，在多任务视觉定位的REC和RES任务上均优于现有方法。
摘要：
多任务视觉定位（MTVG）包含两个子任务，即指代表达理解（REC）和指代表达分割（RES）。现有的代表性方法通常遵循一种研究流程，主要包含三个核心步骤：分别对视觉和语言模态进行独立特征提取、跨模态交互模块，以及针对不同子任务的独立预测头。尽管取得了显著性能，但这条研究路线存在两个局限性：1）语言内容未被充分注入整个视觉主干网络以促进更有效的视觉特征提取，且需要额外的跨模态交互模块；2）REC与RES任务之间的关系未被有效利用以辅助协同预测从而获得更准确的输出。为解决这些问题，本文提出了一种用于多任务视觉定位的渐进式语言引导视觉学习框架，称为PLVL，该框架不仅精细挖掘视觉模态自身固有的特征表达，还逐步注入语言信息以帮助学习与语言相关的视觉特征。通过这种方式，我们的PLVL在充分引入语言引导的同时，无需额外的跨模态融合模块。此外，我们分析指出REC的定位中心在某种程度上有助于识别RES待分割的目标区域。受此启发，我们设计了一个多任务头来完成这两个子任务的协同预测。在多个基准数据集上进行的大量实验全面证实，我们的PLVL在REC和RES任务上均明显优于代表性方法。
英文摘要：
Multi-task visual grounding (MTVG) includes two sub-tasks, i.e., Referring Expression Comprehension (REC) and Referring Expression Segmentation (RES). The existing representative approaches generally follow the research pipeline which mainly consists of three core procedures, including independent feature extraction for visual and linguistic modalities, respectively, cross-modal interaction module, and independent prediction heads for different sub-tasks. Albeit achieving remarkable performance, this research line has two limitations: 1) The linguistic content has not been fully injected into the entire visual backbone for boosting more effective visual feature extraction and it needs an extra cross-modal interaction module; 2) The relationship between REC and RES tasks is not effectively exploited to help the collaborative prediction for more accurate output. To deal with these problems, in this paper, we propose a Progressive Language-guided Visual Learning framework for multi-task visual grounding, called PLVL, which not only finely mine the inherent feature expression of the visual modality itself but also progressively inject the language information to help learn linguistic-related visual features. In this manner, our PLVL does not need additional cross-modal fusion module while fully introducing the language guidance. Furthermore, we analyze that the localization center for REC would help identify the to-be-segmented object region for RES to some extent. Inspired by this investigation, we design a multi-task head to accomplish collaborative predictions for these two sub-tasks. Extensive experiments conducted on several benchmark datasets comprehensively substantiate that our PLVL obviously outperforms the representative methods in both REC and RES tasks. https://github.com/jcwang0602/PLVL
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
