视觉大模型 / VLM

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Wei Zhan, Masayoshi Tomizuka, Mingyu Ding

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts

Qiuhui Chen, Xinyue Hu, Zirui Wang, Yi Hong

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 11 pages, 3 figures

CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes

Maria Parelli, Alexandros Delitzas, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, Thomas Hofmann

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPRW 2023. Code will be made publicly available: https://github.com/AlexDelitzas/3D-VQA

Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT

Mingzhe Hu, Shaoyan Pan, Yuheng Li, Xiaofeng Yang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

CLIPPO: Image-and-Language Understanding from Pixels Only

Michael Tschannen, Basil Mustafa, Neil Houlsby

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2023. Code and pretrained models are available at https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/clippo/README.md

Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning

Yatai Ji, Rongcheng Tu, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, Wei Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2023 accept

Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training

Anthony Meng Huat Tiong, Junnan Li, Boyang Li, Silvio Savarese, Steven C. H. Hoi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments EMNLP 2022 (Findings); correct typos in Equation 2 on page 4

Connecting Vision and Language with Video Localized Narratives

Paul Voigtlaender, Soravit Changpinyo, Jordi Pont-Tuset, Radu Soricut, Vittorio Ferrari

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

Comments Accepted at CVPR 2023

Linearly Mapping from Image to Text Space

Jack Merullo, Louis Castricato, Carsten Eickhoff, Ellie Pavlick

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Comments Accepted at ICLR 2023

Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts

Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, Xiang Wan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Work in progress

OG-SGG: Ontology-Guided Scene Graph Generation. A Case Study in Transfer Learning for Telepresence Robotics

Fernando Amodeo, Fernando Caballero, Natalia Díaz-Rodríguez, Luis Merino

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 20 pages; version accepted and published in IEEE Access

MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering

Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, Mike Zheng Shou

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV

3D Question Answering

Shuquan Ye, Dongdong Chen, Songfang Han, Jing Liao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To Appear at IEEE Transactions on Visualization and Computer Graphics (TVCG) 2022

Look, Read and Ask: Learning to Ask Questions by Reading Text in Images

Soumya Jahagirdar, Shankar Gangisetty, Anand Mishra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Cross-Modal Contrastive Learning for Robust Reasoning in VQA

Qi Zheng, Chaoyue Wang, Daqing Liu, Dadong Wang, Dacheng Tao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Continual VQA for Disaster Response Systems

Aditya Kane, V Manushree, Sahil Khose

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted at Tackling Climate Change with Machine Learning workshop at NeurIPS 2022

ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation

Bin Shan, Yaqian Han, Weichong Yin, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 13 pages, 2 figures

Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision

Tzu-Jui Julius Wang, Jorma Laaksonen, Tomas Langer, Heikki Arponen, Tom E. Bishop

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to WACV'23. Please find supplementary material at https://drive.google.com/file/d/1SmCBGsUgkYLAhmK83RZqY03bq4j3214p/view?usp=sharing

On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization

Shruti Palaskar, Akshita Bhagia, Yonatan Bisk, Florian Metze, Alan W Black, Ana Marasović

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments v2: EMNLP Findings 2022 accepted paper camera-ready version. 9 pages main, 2 pages appendix

Image Semantic Relation Generation

Mingzhe Du

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Fingdings of EMNLP-2022

RepsNet: Combining Vision with Language for Automated Medical Reports

Ajay Kumar Tanwani, Joelle Barral, Daniel Freedman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Journal ref MICCAI 2022, pp. 714--724

Understanding Attention for Vision-and-Language Tasks

Feiqi Cao, Soyeon Caren Han, Siqu Long, Changwei Xu, Josiah Poon

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted in COLING 2022

Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training

Jong Hak Moon, Hyungyung Lee, Woncheol Shin, Young-Hak Kim, Edward Choi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by IEEE Journal of Biomedical and Health Informatics

Journal ref IEEE Journal of Biomedical and Health Informatics 2022

MUST-VQA: MUltilingual Scene-text VQA

Emanuele Vivoli, Ali Furkan Biten, Andres Mafla, Dimosthenis Karatzas, Lluis Gomez

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To be appeared in Text In Everything Workshop in ECCV 2022

An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, Lijuan Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments AAAI 2022 (Oral Presentation)

Pre-training image-language transformers for open-vocabulary tasks

AJ Piergiovanni, Weicheng Kuo, Anelia Angelova

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering

Jiong Wang, Zhou Zhao, Weike Jin

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

An Entropy Clustering Approach for Assessing Visual Question Difficulty

Kento Terao, Toru Tamaki, Bisser Raytchev, Kazufumi Kaneda, Shun'ichi Satoh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Journal ref IEEE Access, Vol.8, pp. 180633-180645, Sep 2020

Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task

Stan Weixian Lei, Difei Gao, Jay Zhangjie Wu, Yuxuan Wang, Wei Liu, Mengmi Zhang, Mike Zheng Shou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 18 pages, 13 figures