arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2412.20206cs.CV

视觉定位:一项综述

Toward Visual Grounding: A Survey

Pengcheng Laboratory(鹏城实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))

Linhui Xiao, Xiaoshan Yang, Xiangyuan Lan, Yaowei Wang, Changsheng Xu

AI总结:

本综述梳理视觉定位的发展与背景,总结近年进展与新挑战,定义规范研究设置,介绍相关数据集与应用,提出未来方向,是该领域最全面的综述,适合不同阶段研究者。

摘要:

视觉定位(也称为指代表达理解和短语定位)旨在基于给定的表达文本,定位图像中的特定区域。该任务模拟视觉与语言模态之间常见的指称关系,使机器具备类人的多模态理解能力,因此在多个领域有广泛应用。然而自2021年以来,视觉定位取得了显著进展,出现了接地预训练、接地多模态大语言模型(grounding multimodal LLMs)、通用视觉定位、千兆像素级定位(giga-pixel grounding)等新概念,带来了诸多新挑战。本综述首先梳理视觉定位的发展历史,概述必要的背景知识;系统追踪并总结该领域的进展,随后精心定义并整理各类设置,以规范未来研究并确保公平比较;此外,深入探究众多相关数据集与应用,重点介绍多个前沿主题;最后,概述视觉定位面临的挑战,提出未来研究的有价值方向,可为后续研究者提供启发。通过提炼共性技术细节,本综述涵盖了过去十年各子主题的代表性工作,据作者所知,这是当前视觉定位领域最全面的综述,适合初学者与资深研究者使用,是理解关键概念、追踪最新研究进展的宝贵资源。作者会在该httpsURL持续追踪相关工作。

英文摘要:

Visual Grounding, also known as Referring Expression Comprehension and Phrase Grounding, aims to ground the specific region(s) within the image(s) based on the given expression text. This task simulates the common referential relationships between visual and linguistic modalities, enabling machines to develop human-like multimodal comprehension capabilities. Consequently, it has extensive applications in various domains. However, since 2021, visual grounding has witnessed significant advancements, with emerging new concepts such as grounded pre-training, grounding multimodal LLMs, generalized visual grounding, and giga-pixel grounding, which have brought numerous new challenges. In this survey, we first examine the developmental history of visual grounding and provide an overview of essential background knowledge. We systematically track and summarize the advancements, and then meticulously define and organize the various settings to standardize future research and ensure a fair comparison. Additionally, we delve into numerous related datasets and applications, and highlight several advanced topics. Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers. By extracting common technical details, this survey encompasses the representative work in each subtopic over the past decade. To the best of our knowledge, this paper represents the most comprehensive overview currently available in the field of visual grounding. This survey is designed to be suitable for both beginners and experienced researchers, serving as an invaluable resource for understanding key concepts and tracking the latest research developments. We keep tracing related work at https://github.com/linhuixiao/Awesome-Visual-Grounding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑