视觉大模型 / VLM

How (not) to ensemble LVLMs for VQA

Lisa Alazraki, Lluis Castrejon, Mostafa Dehghani, Fantine Huot, Jasper Uijlings, Thomas Mensink

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments 4th I Can't Believe It's Not Better Workshop (co-located with NeurIPS 2023)

Global and Local Semantic Completion Learning for Vision-Language Pre-training

Rong-Cheng Tu, Yatai Ji, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, Wei Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2211.13437

Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data

Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding

Junyan Li, Delin Chen, Yining Hong, Zhenfang Chen, Peihao Chen, Yikang Shen, Chuang Gan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Adaptive loose optimization for robust question answering

Jie Ma, Pinghui Wang, Zewei Wang, Dechen Kong, Min Hu, Ting Han, Jun Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 13 pages,8 figures

Reusable Slotwise Mechanisms

Trang Nguyen, Amin Mansouri, Kanika Madan, Khuong Nguyen, Kartik Ahuja, Dianbo Liu, Yoshua Bengio

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Jaeger: A Concatenation-Based Multi-Transformer VQA Model

Jieting Long, Zewei Shi, Penghao Jiang, Yidong Gan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments This paper is the technical research paper of CIKM 2023 DocIU challenges. The authors received the CIKM 2023 DocIU Winner Award, sponsored by Google, Microsoft, and the Centre for data-driven geoscience

A scoping review on multimodal deep learning in biomedical images and texts

Zhaoyi Sun, Mingquan Lin, Qingqing Zhu, Qianqian Xie, Fei Wang, Zhiyong Lu, Yifan Peng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments This paper has been accepted by the Journal of Biomedical Informatics

Journal ref Journal of Biomedical Informatics, Volume 146, October 2023, 104482

Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA

Sheng Zhou, Dan Guo, Jia Li, Xun Yang, Meng Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by TIP 2023

Human Mobility Question Answering (Vision Paper)

Hao Xue, Flora D. Salim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering

Bruno Souza, Marius Aasan, Helio Pedrini, Adín Ramírez Rivera

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To appear in Vision-and-Language Algorithmic Reasoning Workshop at ICCV 2023

Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts

Deniz Engin, Yannis Avrithis

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments ICCV2023 CLVL Workshop (Oral). Project page: https://engindeniz.github.io/vitis

Tackling VQA with Pretrained Foundation Models without Further Training

Alvin De Jun Tan, Bingquan Shen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Pointing out Human Answer Mistakes in a Goal-Oriented Visual Dialogue

Ryosuke Oshima, Seitaro Shinagawa, Hideki Tsunashima, Qi Feng, Shigeo Morishima

专题命中 视觉问答 ：visual language model(abstract)；分类 cs.CV

Comments Accepted at ICCVW 2023

Towards Language-guided Visual Recognition via Dynamic Convolutions

Gen Luo, Yiyi Zhou, Xiaoshuai Sun, Yongjian Wu, Yue Gao, Rongrong Ji

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Generic Attention-model Explainability by Weighted Relevance Accumulation

Yiming Huang, Aozhe Jia, Xiaodan Zhang, Jiawei Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision

Sophia Gu, Christopher Clark, Aniruddha Kembhavi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments website (https://prior.allenai.org/projects/close), code (https://github.com/allenai/close)

PreSTU: Pre-Training for Scene-Text Understanding

Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei-Lun Chao, Radu Soricut

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, Noah A Smith

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

PromptCap: Prompt-Guided Task-Aware Image Captioning

Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, Jiebo Luo

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models

Indranil Sur, Karan Sikka, Matthew Walmer, Kaushik Koneripalli, Anirban Roy, Xiao Lin, Ajay Divakaran, Susmit Jha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Published as conference paper at ICCV 2023. 13 pages, 6 figures, 7 tables

Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories

Thomas Mensink, Jasper Uijlings, Lluis Castrejon, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, André Araujo, Vittorio Ferrari

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ICCV'23

Single Sequence Prediction over Reasoning Graphs for Multi-hop QA

Gowtham Ramesh, Makesh Sreedhar, Junjie Hu

专题命中 视觉问答 ：grounding(abstract)；分类 cs.LG

Diversifying Joint Vision-Language Tokenization Learning

Vardaan Pahuja, AJ Piergiovanni, Anelia Angelova

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to Transformers for Vision (T4V) workshop, CVPR 2023; 7 pages, 5 figures

Scalable Neural-Probabilistic Answer Set Programming

Arseny Skryagin, Daniel Ochs, Devendra Singh Dhami, Kristian Kersting

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 37 pages, 14 figures

Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes

Alexandros Delitzas, Maria Parelli, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, Thomas Hofmann

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments The first two authors contributed equally. arXiv admin note: text overlap with arXiv:2304.06061

An Approach to Solving the Abstraction and Reasoning Corpus (ARC) Challenge

Tan John Chong Min

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 14 pages

PDFVQA: A New Dataset for Real-World VQA on PDF Documents

Yihao Ding, Siwen Luo, Hyunsuk Chung, Soyeon Caren Han

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by ECML-PKDD 2023

Evaluating the Capabilities of Multi-modal Reasoning Models with Synthetic Task Data

Nathan Vaska, Victoria Helus

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Multi-Scale Attention for Audio Question Answering

Guangyao Li, Yixin Xu, Di Hu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments Accepted by InterSpeech 2023