用于图像描述与视觉定位的循环一致性学习
Cycle-Consistency Learning for Captioning and Grounding
AI总结：
提出CyCo循环一致性学习框架，桥接视觉定位与图像描述，实现半弱监督训练并提升全监督性能，生成可描述任意区域的通用描述模型。
摘要：
我们提出，视觉定位与图像描述作为两个相互逆的过程，可以通过精心设计进行桥接，以实现协同训练。基于这一思想，我们引入了CyCo，一个循环一致性学习框架，以改进视觉定位和图像描述的独立训练流程。该框架（1）支持视觉定位的半弱监督训练；（2）提升全监督视觉定位的性能；（3）生成一个能够描述任意图像区域的通用图像描述模型。大量实验表明，我们的全监督定位模型达到了最先进的性能，半弱监督模型也展现出与全监督模型相当的竞争力。我们的图像描述模型能够自由描述图像区域，同时在主流图像描述基准上表现出令人印象深刻的性能。
英文摘要：
We present that visual grounding and image captioning, which perform as two mutually inverse processes, can be bridged together for collaborative training by careful designs. By consolidating this idea, we introduce CyCo, a cyclic-consistent learning framework to ameliorate the independent training pipelines of visual grounding and image captioning. The proposed framework (1) allows the semi-weakly supervised training of visual grounding; (2) improves the performance of fully supervised visual grounding; (3) yields a general captioning model that can describe arbitrary image regions. Extensive experiments show that our fully supervised grounding model achieves state-of-the-art performance, and the semi-weakly supervised one also exhibits competitive performance compared to the fully supervised counterparts. Our image captioning model has the capability to freely describe image regions and meanwhile shows impressive performance on prevalent captioning benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
