Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection. Here, we study algorithms for complex counting questions that involve relationships between objects, attribute identification, reasoning, and more. To do this, we created TallyQA, the world's largest dataset for open-ended counting. We propose a new algorithm for counting that uses relation networks with region proposals. Our method lets relation networks be efficiently used with high-resolution imagery. It yields state-of-the-art results compared to baseline and recent systems on both TallyQA and the HowMany-QA benchmark.
RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answering
School of Computer Science, University of Technology Sydney(技术悉尼大学计算机科学学院);SEDE, University of Technology Sydney(技术悉尼大学SEDE);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
DisasterVQA: A Visual Question Answering Benchmark Dataset for Disaster Scenes
Qatar Computing Research Institute(卡塔尔计算研究所);Hamad Bin Khalifa University(哈马德·本·卡伊夫大学);College of Science & Engineering(科学与工程学院);Qatar University(卡塔尔大学)
Warehouse Spatial Question Answering with LLM Agent
Information Processing Lab, University of Washington, USA(华盛顿大学信息处理实验室);Electronics and Telecommunications Research Institute, South Korea(韩国电子电信研究院);National Center for High-performance Computing, Taiwan(台湾高性能计算国家研究中心)
本文提出一种数据高效的 LLM agent 系统,通过集成多种工具进行空间推理与 API 交互,解决了复杂室内仓库场景下的空间问答任务,并在目标检索、计数和距离估计上实现了高准确率与高效率。
Object Attribute Matters in Visual Question Answering
Jilin University(吉林大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学)
TriViewBench: Controlled Complexity Scaling for Multi-View Structural Reasoning in MLLMs
School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
Visual Enumeration Remains Challenging for Multimodal Generative AI
Department of General Psychology and Department of Mathematics University of Padova(帕多瓦大学心理学系和数学系);Department of General Psychology University of Padova(帕多瓦大学心理学系);Department of General Psychology and Padova Neuroscience Center University of Padova(帕多瓦大学心理学系和帕多瓦神经科学中心);IRCSS San Camillo Hospital, Venice-Lido(威尼斯利多医院IRCSS桑卡莫医院)
BinaryVQA: A Versatile Test Set to Evaluate the Out-of-Distribution Generalization of VQA Models
Quintic AI(昆蒂克人工智能)
我们引入了一个新的测试集的视觉问答(VQA)称为BinaryVQA推动VQA模型的限制。我们的数据集包括1,024张图像中的7,800个问题,涵盖了各种各样的对象、主题和概念。为了简化模型评估,我们只考虑二元问题。问题和答案都是人工精心制定和验证的。大约63%的问题得到了肯定的回答。每个图像的问题数和问题长度的中位数分别为7和5。最先进的OFA模型在BinaryVQA数据集上达到了75%的准确率,这明显低于其在VQA v2测试开发数据集上的性能(94.7%)。我们还沿着几个维度分析了模型行为,包括:a)不同类别的性能,如文本,计数和注视方向,b)模型可解释性,c)问题长度对准确性的影响,d)模型对积极答案的偏见和引入称为ShuffleAcc的新分数,以及e)对拼写和语法错误的敏感性。我们的调查证明了我们的数据集的难度,并表明它可以在未来几年内挑战VQA模型。数据和代码可在以下网址公开获取:DATA and CODE。
Faculty of Computer Science, University of Vienna(维也纳大学计算机学院);UniVie Doctoral School Computer Science, University of Vienna(维也纳大学UniVie计算机博士学院);Georg-August-Universität Göttingen(哥廷根大学);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语言与文化学院)
Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
TomTom(TomTom公司);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Saarland Informatics Campus(萨尔信息学园区)