视觉大模型 / VLM

How good are deep models in understanding the generated images?

Ali Borji

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures

Yongji Wu, Matthew Lentz, Danyang Zhuo, Yao Lu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem

Yudong Han, Liqiang Nie, Jianhua Yin, Jianlong Wu, Yan Yan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 13 pages, 10 figures, submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence

IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer, Siva Reddy, Desmond Elliott, Edoardo Maria Ponti, Ivan Vulić

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ICML 2022

A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA

Yangyang Guo, Liqiang Nie, Yongkang Wong, Yibing Liu, Zhiyong Cheng, Mohan Kankanhalli

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Label-Descriptive Patterns and Their Application to Characterizing Classification Errors

Michael Hedderich, Jonas Fischer, Dietrich Klakow, Jilles Vreeken

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Comments Accepted at ICML 2022

Structured Two-stream Attention Network for Video Question Answering

Lianli Gao, Pengpeng Zeng, Jingkuan Song, Yuan-Fang Li, Wu Liu, Tao Mei, Heng Tao Shen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering

Jiangtong Li, Li Niu, Liqing Zhang

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

Comments To appear in CVPR 2022

Internet-augmented language models through few-shot prompting for open-domain question answering

Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, Nikolai Grigorev

专题命中 视觉问答 ：grounding(abstract)；分类 cs.LG

All You May Need for VQA are Image Captions

Soravit Changpinyo, Doron Kukliansky, Idan Szpektor, Xi Chen, Nan Ding, Radu Soricut

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022)

Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning

Feilong Chen, Xiuyi Chen, Shuang Xu, Bo Xu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ICASSP 2022

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, Di Hu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by CVPR2022 (Oral presentation)

Vision-Language Pre-Training with Triple Contrastive Learning

Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, Junzhou Huang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2022; code: https://github.com/uta-smile/TCL

CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment

Haoyu Song, Li Dong, Wei-Nan Zhang, Ting Liu, Furu Wei

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ACL 2022 main conference

NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks

Fawaz Sammani, Tanmoy Mukherjee, Nikos Deligiannis

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to CVPR 2022

An experimental study of the vision-bottleneck in VQA

Pierre Marza, Corentin Kervadec, Grigory Antipov, Moez Baccouche, Christian Wolf

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

NEWSKVQA: Knowledge-Aware News Video Question Answering

Pranay Gupta, Manish Gupta

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

LaTr: Layout-Aware Transformer for Scene-Text VQA

Ali Furkan Biten, Ron Litman, Yusheng Xie, Srikar Appalaraju, R. Manmatha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams

Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 24 pages; revised version submitted to Language Resources & Evaluation

Journal ref Language Resources and Evaluation 55(3), 2021, pp. 661-688

Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance View

Yangyang Guo, Liqiang Nie, Zhiyong Cheng, Qi Tian, Min Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Multi-Modal Answer Validation for Knowledge-Based VQA

Jialin Wu, Jiasen Lu, Ashish Sabharwal, Roozbeh Mottaghi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments AAAI 2022

MoCA: Incorporating Multi-stage Domain Pretraining and Cross-guided Multimodal Attention for Textbook Question Answering

Fangzhi Xu, Qika Lin, Jun Liu, Lingling Zhang, Tianzhe Zhao, Qi Chai, Yudai Pan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 9 pages, 6 figures

Searching the Search Space of Vision Transformer

Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to NIPS 2021

Structured Multimodal Attentions for TextVQA

Chenyu Gao, Qi Zhu, Peng Wang, Hui Li, Yuliang Liu, Anton van den Hengel, Qi Wu

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV

Comments winner of TextVQA Challenge 2020, Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence

UFO: A UniFied TransfOrmer for Vision-Language Representation Learning

Jianfeng Wang, Xiaowei Hu, Zhe Gan, Zhengyuan Yang, Xiyang Dai, Zicheng Liu, Yumao Lu, Lijuan Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Grounded Graph Decoding Improves Compositional Generalization in Question Answering

Yu Gai, Paras Jain, Wendi Zhang, Joseph E. Gonzalez, Dawn Song, Ion Stoica

专题命中 视觉问答 ：grounding(abstract)；分类 cs.LG

Comments To be published in Findings of EMNLP 2021. Code available at https://github.com/gaiyu0/cfq

Survey of Visual-Semantic Embedding Methods for Zero-Shot Image Retrieval

Kazuya Ueki

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by 20th IEEE International Conference on Machine Learning and Applications (ICMLA2021)

TxT: Crossmodal End-to-End Learning with Transformers

Jan-Martin O. Steitz, Jonas Pfeiffer, Iryna Gurevych, Stefan Roth

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To appear at the 43rd DAGM German Conference on Pattern Recognition (GCPR) 2021

InfographicVQA

Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, C. V Jawahar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Disentangling 3D Prototypical Networks For Few-Shot Concept Learning

Mihir Prabhudesai, Shamit Lal, Darshan Patil, Hsiao-Yu Tung, Adam W Harley, Katerina Fragkiadaki

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV