arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02201cs.CV
收藏

视觉问题中焦点歧义的识别

Acknowledging Focus Ambiguity in Visual Questions

University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Colorado Boulder(科罗拉多大学博尔德分校)

Chongyan Chen, Yu-Yun Tseng, Zhuoheng Li, Anush Venkatesh, Danna Gurari

AI总结:

针对现有视觉问答研究未考虑问题焦点歧义的问题,提出首个标注所有合理焦点区域的VQ-FocusAmbiguity数据集,测试了现代模型在两项相关新任务上的表现并公开数据集以推动研究。

摘要:

目前已发表的视觉问答(VQA)研究均未考虑问题描述内容在图像中位置的歧义问题。为填补这一空白,我们提出了VQ-FocusAmbiguity,这是首个为每个可得到有效答案的问题所指代的所有合理图像区域提供视觉定位的VQA数据集。随后我们将该数据集与现有数据集进行分析对比,揭示了其独特属性。最后,我们针对两个与识别焦点歧义相关的新任务对现代模型进行基准测试:识别视觉问题是否存在焦点歧义,以及定位图像中所有合理的焦点区域。结果表明,该数据集对现代模型具有挑战性。为推动这些任务的未来研究,我们公开了该数据集及评估服务器,网址为https://vizwiz.org/tasks-and-datasets/focus-ambiguity-in-visual-questions。

英文摘要:

No published work on visual question answering (VQA) accounts for ambiguity regarding where the content described in the question is located in the image. To fill this gap, we introduce VQ-FocusAmbiguity, the first VQA dataset that visually grounds each plausible image region a question could refer to when arriving at valid answers. We next analyze and compare our dataset to existing datasets to reveal its unique properties. Finally, we benchmark modern models for two novel tasks related to acknowledging focus ambiguity: recognizing whether a visual question has focus ambiguity and locating all plausible focus regions within the image. Results show that the dataset is challenging for modern models. To facilitate future progress on these tasks, we publicly share the dataset with an evaluation server at https://vizwiz.org/tasks-and-datasets/focus-ambiguity-in-visual-questions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑