视觉大模型 / VLM - arXivDaily 专题

2406.04236 2024-06-07 cs.CV 70%

Understanding Information Storage and Transfer in Multi-modal Large Language Models

Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, Daniela Massiceti

专题命中视觉问答：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 20 pages

2405.16886 2024-05-28 cs.CV 70%

Hawk: Learning to Understand Open-World Video Anomalies

Jiaqi Tang, Hao Lu, Ruizheng Wu, Xiaogang Xu, Ke Ma, Cheng Fang, Bin Guo, Jiangbo Lu, Qifeng Chen, Ying-Cong Chen

专题命中视觉问答：VLM(abstract)；visual language model(abstract)；分类 cs.CV

2402.09181 2024-04-23 eess.IV cs.CV 70%

OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM

Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, Ping Luo

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

2312.03587 2024-04-04 cs.CV 70%

Language-Informed Visual Concept Learning

Sharon Lee, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments ICLR 2024. The first two authors contributed equally and are alphabetically ordered. Project page: https://ai.stanford.edu/~yzzhang/projects/concept-axes/

2403.15456 2024-04-01 cs.AI cs.CL 70%

WoLF: Wide-scope Large Language Model Framework for CXR Understanding

Seil Kang, Donghyun Kim, Junhyeok Kim, Hyo Kyung Lee, Seong Jae Hwang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.AI

Comments 11 pages main paper, 2 pages supplementary

2310.08825 2024-03-11 cs.CV 70%

From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Dongsheng Jiang, Yuchen Liu, Songlin Liu, Jin'e Zhao, Hao Zhang, Zhen Gao, Xiaopeng Zhang, Jin Li, Hongkai Xiong

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2302.00389 2024-03-04 cs.AI 70%

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, Shangsong Liang

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.AI

2402.05138 2024-02-09 cs.AI cs.CL 70%

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang, Jiajun Jiao, Renjie Pi, Jipeng Zhang, Xiangliang Zhang

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Work in progress

2312.14074 2023-12-22 cs.CV 70%

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

Senqiao Yang, Jiaming Liu, Ray Zhang, Mingjie Pan, Zoey Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Yandong Guo, Shanghang Zhang

专题命中视觉问答：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

2312.06352 2023-12-12 cs.CV cs.CL 70%

NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations

Yuichi Inoue, Yuki Yada, Kotaro Tanahashi, Yu Yamaguchi

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted at LLVM-AD Workshop @ WACV 2024

2305.12397 2023-12-11 cs.CV 70%

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

Yuanyuan Jiang, Jianqin Yin

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Accepted to EMNLP 2023 Findings

2311.17043 2023-11-29 cs.CV cs.CL 70%

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

Yanwei Li, Chengyao Wang, Jiaya Jia

专题命中视觉问答：vision language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Code is available at https://github.com/dvlab-research/LLaMA-VID

2310.09478 2023-11-08 cs.CV 70%

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, Mohamed Elhoseiny

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2310.19145 2023-10-31 cs.CL cs.CV 70%

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Tuhin Chakrabarty, Kanishk Singh, Arkadiy Saakyan, Smaranda Muresan

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Findings of EMNLP 2023 (Long paper)

2310.19061 2023-10-31 cs.CV 70%

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

Zhiling Yan, Kai Zhang, Rong Zhou, Lifang He, Xiang Li, Lichao Sun

专题命中视觉问答：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV

2307.00716 2023-10-31 cs.CV 70%

JourneyDB: A Benchmark for Generative Image Understanding

Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Limin Wang, Hongsheng Li

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

2310.17050 2023-10-27 cs.CV 70%

Exploring Question Decomposition for Zero-Shot VQA

Zaid Khan, Vijay Kumar BG, Samuel Schulter, Manmohan Chandraker, Yun Fu

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments NeurIPS 2023 Camera Ready

2308.09033 2023-09-20 cs.CV 70%

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Fawaz Sammani, Nikos Deligiannis

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted to ICCVW 2023

2308.10354 2023-08-22 cs.AI cs.CL 70%

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Zeinab Sadat Taghavi, Soroush Gooran, Seyed Arshan Dalili, Hamidreza Amirzadeh, Mohammad Jalal Nematbakhsh, Hossein Sameti

专题命中视觉问答：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments 18 pages,

2307.09416 2023-07-20 cs.CV cs.CL 70%

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted as oral at ACM MultiMedia 2023 (Brave New Ideas track)

2205.12005 2023-07-06 cs.CL cs.CV 70%

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou, Luo Si

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Journal ref EMNLP2022

2305.16328 2023-05-29 cs.CL cs.LG 70%

Semantic Composition in Visually Grounded Language Models

Rohan Pandey

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.LG

Comments Carnegie Mellon University Senior Thesis. arXiv admin note: substantial text overlap with arXiv:2212.10549

2303.02635 2023-03-07 cs.CV cs.MM 70%

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Kang Chen, Xiangqian Wu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

2208.09374 2022-08-22 cs.CV 70%

VLMAE: Vision-Language Masked Autoencoder

Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, Bo Ren

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

2208.03030 2022-08-08 cs.CL cs.CV 70%

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Bingning Wang, Feiyang Lv, Ting Yao, Yiming Yuan, Jin Ma, Yu Luo, Haijin Liang

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments CIKM2022 camera ready version

2201.10963 2022-05-31 cs.CV cs.MM 70%

Learning to Compose Diversified Prompts for Image Emotion Classification

Sinuo Deng, Lifang Wu, Ge Shi, Lehao Xing, Meng Jian, Ye Xiang

专题命中视觉问答：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments 10 pages, 5 figures

2106.13488 2021-11-10 cs.CV 70%

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2021

2108.08965 2021-08-23 cs.CV cs.CL 70%

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Xiaopeng Lu, Zhen Fan, Yansen Wang, Jean Oh, Carolyn P. Rose

专题命中视觉问答：visual question answering(abstract)；grounding(abstract)；分类 cs.CV

Comments 9 pages

1911.09655 2019-11-22 cs.CL cs.LG cs.SD eess.AS 70%

Temporal Reasoning via Audio Question Answering

Haytham M. Fayek, Justin Johnson

专题命中视觉问答：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.LG

1. 视觉问答 3148 篇

Understanding Information Storage and Transfer in Multi-modal Large Language Models

Hawk: Learning to Understand Open-World Video Anomalies

OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM

Language-Informed Visual Concept Learning

WoLF: Wide-scope Large Language Model Framework for CXR Understanding

From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations

Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarios

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

JourneyDB: A Benchmark for Generative Image Understanding

Exploring Question Decomposition for Zero-Shot VQA

Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

Semantic Composition in Visually Grounded Language Models

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

Champion Solution for the WSDM2023 Toloka VQA Challenge

VLMAE: Vision-Language Masked Autoencoder

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

Learning to Compose Diversified Prompts for Image Emotion Classification

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training

Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling

Temporal Reasoning via Audio Question Answering