arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2607.04638cs.CV

用于弱监督指称表达式理解的学习结构化视觉组合表示

Learning Structured Visual Compositional Representations for Weakly Supervised Referring Expression Comprehension

The University of Western Australia(西澳大利亚大学);Murdoch University(莫道克大学);Sun Yat-sen University(中山大学);The Hong Kong University of Science and Technology(香港科技大学)

Lian Xu, Mohammed Bennamoun, Farid Boussaid, Hamid Laga, Yulan Guo, Dan Xu

AI总结:

针对弱监督指称表达式理解问题,提出结构化视觉组合表示学习框架,显式建模一元对象嵌入和成对关系嵌入,引入组合对齐机制,实验证明该方法有效。

摘要:

指称表达式理解旨在定位自然语言描述图像中的对象。在弱监督指称表达式理解中,现有方法主要在锚点级视觉表示上操作,关系交互隐式编码在单个锚点特征中。本文提出用于弱监督指称表达式理解的结构化视觉组合表示(SVCR)学习框架,显式建模一元对象嵌入和成对关系嵌入,形成结构化视觉表示空间,引入组合对齐机制,在多个数据集上实验表明该方法取得了最优性能。

英文摘要:

Referring expression comprehension (REC) aims to localize the object in an image described by natural language. In Weakly supervised REC (WREC), existing approaches primarily operate on anchor-level visual representations. Even when enriched with auxiliary cues, relational interactions remain implicitly encoded within individual anchor features. The resulting visual representation remains flat and unary-only, limiting its ability to align with the structured nature of language. In this work, we propose a Structured Visual Compositional Representation (SVCR) learning framework for WREC. Rather than implicitly encoding relations within unary anchors, the proposed SVCR explicitly models both unary object embeddings and pairwise relational embeddings, forming a structured visual representation space. We further introduce a compositional alignment mechanism that matches unary and pairwise visual representations with their corresponding textual embeddings in a unified manner, enabling compositional visual-textual matching under weak supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that the proposed SVCR achieves state-of-the-art performance. These results demonstrate the effectiveness of explicit structured visual representations and visual-textual alignment for WREC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑