视觉大模型 / VLM - arXivDaily 专题

2210.07179 2023-03-16 cs.CV cs.AI cs.CL cs.LG 67%

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

Oscar Mañas, Pau Rodriguez, Saba Ahmadi, Aida Nematzadeh, Yash Goyal, Aishwarya Agrawal

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at EACL 2023 (main track); 26 pages, 21 figures, 6 tables; Pau Rodriguez and Saba Ahmadi had equal contributions

2210.05861 2023-01-24 cs.CV cs.AI cs.LG 67%

SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models

Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, Animesh Garg

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICLR 2023. Project page: https://slotformer.github.io/

2201.05729 2023-01-02 cs.CV cs.AI cs.CL cs.LG cs.MM 67%

CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks

Zhecan Wang, Noel Codella, Yen-Chun Chen, Luowei Zhou, Jianwei Yang, Xiyang Dai, Bin Xiao, Haoxuan You, Shih-Fu Chang, Lu Yuan

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This paper is greatly modified and updated to be re-submitted to another conference. The new paper is under the name "Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks", https://doi.org/10.48550/arXiv.2204.10496

1712.05474 2022-08-29 cs.CV cs.AI cs.LG 67%

AI2-THOR: An Interactive 3D Environment for Visual AI

Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, Aniruddha Kembhavi, Abhinav Gupta, Ali Farhadi

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

2203.17247 2022-08-24 cs.CV cs.AI cs.CL cs.LG 67%

VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers

Estelle Aflalo, Meng Du, Shao-Yen Tseng, Yongfei Liu, Chenfei Wu, Nan Duan, Vasudev Lal

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Best Demo Award at CVPR 2022

2207.00691 2022-07-05 cs.CY cs.AI cs.CL cs.CV cs.LG 67%

American == White in Multimodal Language-and-Image AI

Robert Wolfe, Aylin Caliskan

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to AI Ethics and Society 2022

2104.07921 2022-06-14 cs.CV cs.AI cs.CL cs.LG 67%

VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks

Hung Le, Nancy F. Chen, Steven C. H. Hoi

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at NAACL 2022 (Oral)

2204.10496 2022-04-29 cs.CV cs.AI cs.CL cs.LG cs.MM 67%

Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks

Zhecan Wang, Noel Codella, Yen-Chun Chen, Luowei Zhou, Xiyang Dai, Bin Xiao, Jianwei Yang, Haoxuan You, Kai-Wei Chang, Shih-fu Chang, Lu Yuan

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2201.05729

2104.00743 2022-04-21 cs.CV cs.AI cs.CL cs.LG 67%

Towards General Purpose Vision Systems

Tanmay Gupta, Amita Kamath, Aniruddha Kembhavi, Derek Hoiem

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2022 Oral; Project page: https://prior.allenai.org/projects/gpv

2204.00879 2022-04-05 cs.CL 67%

Co-VQA : Answering by Interactive Sub Question Sequence

Ruonan Wang, Yuxi Qian, Fangxiang Feng, Xiaojie Wang, Huixing Jiang

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)

Comments Accepted by Findings of ACL 2022

2109.00590 2022-03-29 cs.CL cs.AI cs.CV cs.LG 67%

WebQA: Multihop and Multimodal QA

Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, Yonatan Bisk

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR Camera ready

2108.08217 2021-08-19 cs.CV cs.AI cs.CL cs.LG cs.MM 67%

X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics

Yehao Li, Yingwei Pan, Jingwen Chen, Ting Yao, Tao Mei

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by 2021 ACMMM Open Source Software Competition. Source code: https://github.com/YehLi/xmodaler

2107.06383 2021-07-15 cs.CV cs.AI cs.CL cs.LG 67%

How Much Can CLIP Benefit Vision-and-Language Tasks?

Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, Kurt Keutzer

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 14 pages

2102.02779 2021-05-25 cs.CL cs.AI cs.CV cs.LG 67%

Unifying Vision-and-Language Tasks via Text Generation

Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2021 (15 pages, 4 figures, 14 tables)

2105.00421 2021-05-04 cs.CV cs.AI cs.LG 67%

A survey on VQA_Datasets and Approaches

Yeyun Zou, Qiyu Xie

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 10 pages

2012.03678 2020-12-08 cs.CV cs.AI cs.CL cs.LG 67%

Generating Natural Questions from Images for Multimodal Assistants

Alkesh Patel, Akanksha Bindal, Hadas Kotek, Christopher Klein, Jason Williams

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 4 pages, 1 reference page, 5 figures, 4 tables

2011.11735 2020-11-25 cs.AI cs.CV cs.LG 67%

Large Scale Multimodal Classification Using an Ensemble of Transformer Models and Co-Attention

Varnith Chordia, Vijay Kumar BG

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

1908.01801 2020-07-23 cs.CV cs.AI cs.CL cs.LG 67%

Answering Questions about Data Visualizations using Efficient Bimodal Fusion

Kushal Kafle, Robik Shrestha, Brian Price, Scott Cohen, Christopher Kanan

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Presented at WACV, 2020

1911.03977 2020-07-15 cs.AI cs.CL cs.CV cs.LG 67%

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

Chao Zhang, Zichao Yang, Xiaodong He, Li Deng

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

1904.05880 2020-03-10 cs.CV cs.AI cs.CL cs.IR cs.LG 67%

Factor Graph Attention

Idan Schwartz, Seunghak Yu, Tamir Hazan, Alexander Schwing

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2019; revised version includes bottom-up features

1905.13211 2020-02-18 cs.LG cs.AI cs.CV cs.NE stat.ML 67%

What Can Neural Networks Reason About?

Keyulu Xu, Jingling Li, Mozhi Zhang, Simon S. Du, Ken-ichi Kawarabayashi, Stefanie Jegelka

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

1912.09551 2019-12-23 cs.CV cs.AI cs.CL cs.LG 67%

Deep Exemplar Networks for VQA and VQG

Badri N. Patro, Vinay P. Namboodiri

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This work is an extension of CVPR-2018 accepted paper arXiv:1804.00298 and EMNLP-2018 accepted paper arXiv:1808.03986

1610.02391 2019-12-04 cs.CV cs.AI cs.LG 67%

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This version was published in International Journal of Computer Vision (IJCV) in 2019; A previous version of the paper was published at International Conference on Computer Vision (ICCV'17)

1908.06336 2019-10-24 cs.CV cs.AI cs.CL cs.LG 67%

What is needed for simple spatial language capabilities in VQA?

Alexander Kuhnle, Ann Copestake

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

1904.12584 2019-04-30 cs.CV cs.AI cs.CL cs.LG 67%

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision

Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2019 (Oral). Project page: http://nscl.csail.mit.edu/

1810.02338 2019-01-16 cs.AI cs.CL cs.CV cs.LG 67%

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments NeurIPS 2018 (spotlight). The first two authors contributed equally to this work. Project page: http://nsvqa.csail.mit.edu

1805.07932 2018-10-22 cs.CV cs.AI cs.CL cs.LG 67%

Bilinear Attention Networks

Jin-Hwa Kim, Jaehyun Jun, Byoung-Tak Zhang

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by NIPS 2018; Figure 1 was updated

1806.04284 2018-06-13 cs.CL cs.AI cs.CV cs.LG cs.MM 67%

iParaphrasing: Extracting Visually Grounded Paraphrases via an Image

Chenhui Chu, Mayu Otani, Yuta Nakashima

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments COLING 2018

1706.01322 2018-05-15 cs.CL cs.AI cs.CV cs.LG 67%

Deep learning evaluation using deep linguistic processing

Alexander Kuhnle, Ann Copestake

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

1712.06228 2017-12-19 cs.CV cs.AI cs.LG 67%

Visual Explanations from Hadamard Product in Multimodal Deep Networks

Jin-Hwa Kim, Byoung-Tak Zhang

专题命中视觉问答：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 8 pages, 5 figures, including appendix, NIPS 2017 Workshop on Visually-Grounded Interaction and Language (ViGIL)

1. 视觉问答 3164 篇

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models

CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks

AI2-THOR: An Interactive 3D Environment for Visual AI

VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers

American == White in Multimodal Language-and-Image AI

VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks

Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks

Towards General Purpose Vision Systems

Co-VQA : Answering by Interactive Sub Question Sequence

WebQA: Multihop and Multimodal QA

X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics

How Much Can CLIP Benefit Vision-and-Language Tasks?

Unifying Vision-and-Language Tasks via Text Generation

A survey on VQA_Datasets and Approaches

Generating Natural Questions from Images for Multimodal Assistants

Large Scale Multimodal Classification Using an Ensemble of Transformer Models and Co-Attention

Answering Questions about Data Visualizations using Efficient Bimodal Fusion

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

Factor Graph Attention

What Can Neural Networks Reason About?

Deep Exemplar Networks for VQA and VQG

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

What is needed for simple spatial language capabilities in VQA?

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

Bilinear Attention Networks

iParaphrasing: Extracting Visually Grounded Paraphrases via an Image

Deep learning evaluation using deep linguistic processing

Visual Explanations from Hadamard Product in Multimodal Deep Networks