AlignCAT: 类别与属性的视觉-语言对齐用于弱监督视觉定位
AlignCAT: Visual-Linguistic Alignment of Category and Attribute for Weakly Supervised Visual Grounding
AI总结：
AlignCAT通过粗粒度和细粒度对齐模块，利用类别与属性信息增强视觉-语言匹配，在三个基准上超越现有弱监督视觉定位方法。
摘要：
弱监督视觉定位（VG）旨在基于文本描述定位图像中的对象。尽管取得了显著进展，现有方法缺乏强大的跨模态推理能力，以区分文本表达中因类别和属性歧义而产生的细微语义差异。为解决这些挑战，我们引入了AlignCAT，一种新颖的基于查询的语义匹配框架，用于弱监督VG。为增强视觉-语言对齐，我们提出了一个粗粒度对齐模块，利用类别信息和全局上下文，有效减轻来自类别不一致对象的干扰。随后，一个细粒度对齐模块利用描述性信息并捕获词级文本特征以实现属性一致性。通过充分利用语言线索，我们提出的AlignCAT逐步过滤掉未对齐的视觉查询，并增强对比学习效率。在三个VG基准（即RefCOCO、RefCOCO+和RefCOCOg）上的大量实验验证了AlignCAT在两项VG任务上优于现有弱监督方法。我们的代码可在以下网址获取：https://github.com/I2-Multimedia-Lab/AlignCAT。
英文摘要：
Weakly supervised visual grounding (VG) aims to locate objects in images based on text descriptions. Despite significant progress, existing methods lack strong cross-modal reasoning to distinguish subtle semantic differences in text expressions due to category-based and attribute-based ambiguity. To address these challenges, we introduce AlignCAT, a novel query-based semantic matching framework for weakly supervised VG. To enhance visual-linguistic alignment, we propose a coarse-grained alignment module that utilizes category information and global context, effectively mitigating interference from category-inconsistent objects. Subsequently, a fine-grained alignment module leverages descriptive information and captures word-level text features to achieve attribute consistency. By exploiting linguistic cues to their fullest extent, our proposed AlignCAT progressively filters out misaligned visual queries and enhances contrastive learning efficiency. Extensive experiments on three VG benchmarks, namely RefCOCO, RefCOCO+, and RefCOCOg, verify the superiority of AlignCAT against existing weakly supervised methods on two VG tasks. Our code is available at: https://github.com/I2-Multimedia-Lab/AlignCAT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
