视觉大模型 / VLM

Evaluating Image Hallucination in Text-to-Image Generation with Question-Answering

Youngsun Lim, Hojun Choi, Hyunjung Shim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 20 pages

SIMPLOT: Enhancing Chart Question Answering by Distilling Essentials

Wonjoong Kim, Sangwu Park, Yeonjun In, Seokwon Han, Chanyoung Park

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Findings of NAACL 2025

Large Models in Dialogue for Active Perception and Anomaly Detection

Tzoulio Chamiti, Nikolaos Passalis, Anastasios Tefas

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to International Conference of Pattern Recognition (ICPR 2024)

RAMQA: A Unified Framework for Retrieval-Augmented Multi-Modal Question Answering

Yang Bai, Christan Earl Grant, Daisy Zhe Wang

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.AI、cs.LG

Comments Accepted by NAACL 2025 Findings

SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers

Shraman Pramanick, Rama Chellappa, Subhashini Venugopalan

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024, Datasets & Benchmarks track

Long Story Short: Story-level Video Understanding from 20K Short Films

Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes

Koen Kraaijveld, Yifan Jiang, Kaixin Ma, Filip Ilievski

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 10 figures, accepted to AAAI-25

Damage Assessment after Natural Disasters with UAVs: Semantic Feature Extraction using Deep Learning

Nethmi S. Hewawiththi, M. Mahesha Viduranga, Vanodhya G. Warnasooriya, Tharindu Fernando, Himal A. Suraweera, Sridha Sridharan, Clinton Fookes

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 11 pages, 8 figures

Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering

Sai Bhargav Rongali, Mohamad Hassan N C, Ankit Jha, Neha Bhargava, Saurabh Prasad, Biplab Banerjee

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV、cs.AI

Journal ref WACV2025

How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey

Yayun Qi, Hongxi Li, Yiqi Song, Xinxiao Wu, Jiebo Luo

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Under Review

Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models

Hao Yi, Qingyang Li, Yulan Hu, Fuzheng Zhang, Di Zhang, Yong Liu

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs

Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, Zhiqiang Shen

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Camera-ready Version. Website at https://mbzuai-llm.github.io/webpage2code/

StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, Maosong Sun

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Designing a Robust Radiology Report Generation System

Sonit Singh

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 21 pages, 2 figures

Nearest Neighbor Normalization Improves Multimodal Retrieval

Neil Chowdhury, Franklin Wang, Sumedh Shenoy, Douwe Kiela, Sarah Schwettmann, Tristan Thrush

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Journal ref EMNLP 2024

Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP

Chen Huang, Skyler Seto, Samira Abnar, David Grangier, Navdeep Jaitly, Josh Susskind

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2024

Kvasir-VQA: A Text-Image Pair GI Tract Dataset

Sushant Gautam, Andrea Storås, Cise Midoglu, Steven A. Hicks, Vajira Thambawita, Pål Halvorsen, Michael A. Riegler

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments to be published in VLM4Bio 2024, part of the ACM Multimedia (ACM MM) conference 2024

Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering

He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perception

Junxiao Shen, John Dudley, Per Ola Kristensson

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV、cs.AI

HAMMR: HierArchical MultiModal React agents for generic VQA

Lluis Castrejon, Thomas Mensink, Howard Zhou, Vittorio Ferrari, Andre Araujo, Jasper Uijlings

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge

Haibo Wang, Weifeng Ge

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV'24

Core Tokensets for Data-efficient Sequential Training of Transformers

Subarnaduti Paul, Manuel Brack, Patrick Schramowski, Kristian Kersting, Martin Mundt

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Attribute Diversity Determines the Systematicity Gap in VQA

Ian Berlot-Attwell, Kumar Krishna Agrawal, A. Michael Carrell, Yash Sharma, Naomi Saphra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Comments 36 pages, 27 figures, EMNLP 2024

A comprehensive study of on-device NLP applications -- VQA, automated Form filling, Smart Replies for Linguistic Codeswitching

Naman Goyal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI、cs.LG

Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Models

Saman Motamed, Danda Pani Paudel, Luc Van Gool

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Explore the Hallucination on Low-level Perception for MLLMs

Yinan Sun, Zicheng Zhang, Haoning Wu, Xiaohong Liu, Weisi Lin, Guangtao Zhai, Xiongkuo Min

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Top-down Activity Representation Learning for Video Question Answering

Yanan Wang, Shuichiro Haruta, Donghuo Zeng, Julio Vizcarra, Mori Kurokawa

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Comments presented at MIRU2024

EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysis

Danli Shi, Weiyi Zhang, Jiancheng Yang, Siyu Huang, Xiaolan Chen, Mayinuer Yusufu, Kai Jin, Shan Lin, Shunming Liu, Qing Zhang, Mingguang He

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models

Bin Fu, Qiyang Wan, Jialin Li, Ruiping Wang, Xilin Chen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments 39 pages, 28 figures, 4 tables. Accepted at The 35th British Machine Vision Conference (BMVC 2024). Project page at https://fubin29.github.io/Blocks-as-Probes/

Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models

Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, Xiang Bai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024 Highlight