视觉大模型 / VLM

Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?

Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, Ming-Wei Chang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2023 (main conference); Our dataset and evaluation is available at https://open-vision-language.github.io/infoseek/

Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI

Leila Arras, Ahmed Osman, Wojciech Samek

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 37 pages, 9 tables, 2 figures (plus appendix 14 pages)

Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting

Chantal Pellegrini, Matthias Keicher, Ege Özsoy, Nassir Navab

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments accepted at MICCAI 2023

Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP

Vedant Palit, Rohan Pandey, Aryaman Arora, Paul Pu Liang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Final version for 5th Workshop on Closing the Loop Between Vision and Language (CLVL) @ ICCV 2023. 4 pages, 5 figures

TeCH: Text-guided Reconstruction of Lifelike Clothed Humans

Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Jiaxiang Tang, Deng Cai, Justus Thies

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Project: https://huangyangyi.github.io/TeCH, Code: https://github.com/huangyangyi/TeCH

CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery

Long Bai, Mobarakol Islam, Hongliang Ren

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments To appear in MICCAI 2023. Code availability: https://github.com/longbai1006/CAT-ViL

Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images

Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, Roy Schwartz

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICCV 2023. Website: whoops-benchmark.github.io

Foundation Model is Efficient Multimodal Multitask Model Selector

Fanqing Meng, Wenqi Shao, Zhanglin Peng, Chonghe Jiang, Kaipeng Zhang, Yu Qiao, Ping Luo

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner

Zikang Liu, Sihan Chen, Longteng Guo, Handong Li, Xingjian He, Jing Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 12 pages. Accepted by ACM MM '23

A reinforcement learning approach for VQA validation: an application to diabetic macular edema grading

Tatiana Fountoukidou, Raphael Sznitman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 16 pages (+ 23 pages supplementary material)

Journal ref Medical image analysis 87 (2023): 102822

Compositionality as Lexical Symmetry

Ekin Akyürek, Jacob Andreas

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments ACL2023 Final Version

Switch-BERT: Learning to Model Multimodal Interactions by Switching Attention and Input

Qingpei Guo, Kaisheng Yao, Wei Chu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV2022

Q2ATransformer: Improving Medical VQA via an Answer Querying Decoder

Yunyi Liu, Zhanyu Wang, Dong Xu, Luping Zhou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models

Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip Ilievski

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 5 figures, 7 tables

i-Code Studio: A Configurable and Composable Framework for Integrative AI

Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Combo of Thinking and Observing for Outside-Knowledge VQA

Qingyi Si, Yuchen Mo, Zheng Lin, Huishan Ji, Weiping Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments ACL-23, Main Conference

Multimodal Federated Learning via Contrastive Representation Ensemble

Qiying Yu, Yang Liu, Yimu Wang, Ke Xu, Jingjing Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2023, update

Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA

Yongxin Zhu, Zhen Liu, Yukang Liang, Xin Li, Hao Liu, Changcun Bao, Linli Xu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments accepted in AAAI 2023

VQA with Cascade of Self- and Co-Attention Blocks

Aakansha Mishra, Ashish Anand, Prithwijit Guha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

MQA: Answering the Question via Robotic Manipulation

Yuhong Deng, Di Guo, Xiaofeng Guo, Naifu Zhang, Huaping Liu, Fuchun Sun

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments has been accepted by Robotics: Science and Systems 2021

Journal ref Robotics: Science and System (RSS2021)

BinaryVQA: A Versatile Test Set to Evaluate the Out-of-Distribution Generalization of VQA Models

Ali Borji

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

CPL: Counterfactual Prompt Learning for Vision and Language Models

Xuehai He, Diji Yang, Weixi Feng, Tsu-Jui Fu, Arjun Akula, Varun Jampani, Pradyumna Narayana, Sugato Basu, William Yang Wang, Xin Eric Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Visual Named Entity Linking: A New Dataset and A Baseline

Wenxiang Sun, Yixing Fan, Jiafeng Guo, Ruqing Zhang, Xueqi Cheng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 11 figures, published to EMNLP 2022(findings)

TVLT: Textless Vision-Language Transformer

Zineng Tang, Jaemin Cho, Yixin Nie, Mohit Bansal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2022 Oral (21 pages; the first three authors contributed equally)

WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models

Yonatan Bitton, Nitzan Bitton Guetta, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2022, Datasets and Benchmarks. Website: https://winogavil.github.io/

LAVIS: A Library for Language-Vision Intelligence

Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, Steven C. H. Hoi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Preprint of LAVIS technical report

Sparse Continuous Distributions and Fenchel-Young Losses

André F. T. Martins, Marcos Treviso, António Farinhas, Pedro M. Q. Aguiar, Mário A. T. Figueiredo, Mathieu Blondel, Vlad Niculae

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Comments JMLR 2022 camera ready version. arXiv admin note: text overlap with arXiv:2006.07214

Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?

Patrick Schramowski, Christopher Tauchmann, Kristian Kersting

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2110.04222

Less Is More: Linear Layers on CLIP Features as Powerful VizWiz Model

Fabian Deuser, Konrad Habel, Philipp J. Rösch, Norbert Oswald

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments VizWiz Grand Challenge: Describing Images and Videos Taken by Blind People (CVPR Workshop 2022)

V-Doc : Visual questions answers with Documents

Yihao Ding, Zhe Huang, Runlin Wang, Yanhang Zhang, Xianru Chen, Yuzhong Ma, Hyunsuk Chung, Soyeon Caren Han

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2022