视觉大模型 / VLM

MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning

Pu Yang, Bin Dong

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Consistency of Compositional Generalization across Multiple Levels

Chuanhao Li, Zhen Li, Chenchen Jing, Xiaomeng Fan, Wenbo Ye, Yuwei Wu, Yunde Jia

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI 2025

CEGI: Measuring the trade-off between efficiency and carbon emissions for SLMs and VLMs

Abhas Kumar, Kapil Pathak, Rajesh Kavuru, Prabhakar Srinivasan

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)；分类 cs.AI、cs.LG

Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy

Te Yang, Jian Jia, Xiangyu Zhu, Weisong Zhao, Bo Wang, Yanhua Cheng, Yan Li, Shengyuan Liu, Quan Chen, Peng Jiang, Kun Gai, Zhen Lei

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answering

Guohao Sun, Can Qin, Huazhu Fu, Linwei Wang, Zhiqiang Tao

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments EMNLP 2024 Main conference

Mind the Uncertainty in Human Disagreement: Evaluating Discrepancies between Model Predictions and Human Responses in VQA

Jian Lan, Diego Frassinelli, Barbara Plank

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Parameter Efficient Reinforcement Learning from Human Feedback

Hakim Sidahmed, Samrat Phatale, Alex Hutcheson, Zhuonan Lin, Zhang Chen, Zac Yu, Jarvis Jin, Simral Chaudhary, Roman Komarytsia, Christiane Ahlheim, Yonghao Zhu, Bowen Li, Saravanan Ganesh, Bill Byrne, Jessica Hoffmann, Hassan Mansoor, Wei Li, Abhinav Rastogi, Lucas Dixon

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.AI、cs.LG

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Paper published at AAAI 2024 Spring Symposium Series

Extracting Training Data from Document-Based VQA Models

Francesco Pinto, Nathalie Rauschmayr, Florian Tramèr, Philip Torr, Federico Tombari

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments ICML 2024

WeatherQA: Can Multimodal Language Models Reason about Severe Weather?

Chengqian Ma, Zhanxiang Hua, Alexandra Anderson-Frey, Vikram Iyer, Xin Liu, Lianhui Qin

专题命中 视觉问答 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 26 pages, 9 figures

AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding

Alessandro Suglia, Claudio Greco, Katie Baker, Jose L. Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Code available https://github.com/alanaai/EVUD

Wings: Learning Multimodal LLMs without Text-only Forgetting

Yi-Kai Zhang, Shiyin Lu, Yang Li, Yanqing Ma, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Bridging Vision and Language Spaces with Assignment Prediction

Jungin Park, Jiyoung Lee, Kwanghoon Sohn

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments ICLR 2024 Camera-ready

Can I Trust Your Answer? Visually Grounded Video Question Answering

Junbin Xiao, Angela Yao, Yicong Li, Tat Seng Chua

专题命中 视觉问答 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR'24. (Compared with preprint version, we mainly improve the presentation, discuss more related works, and extend experiments in Appendix.)

A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis

Yingshu Li, Yunyi Liu, Zhanyu Wang, Xinyu Liang, Lei Wang, Lingqiao Liu, Leyang Cui, Zhaopeng Tu, Longyue Wang, Luping Zhou

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

MIVC: Multiple Instance Visual Component for Visual-Language Models

Wenyi Wu, Qi Li, Wenliang Zhong, Junzhou Huang

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted at WACV 2024

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs

Shengzhi Li, Nima Tajbakhsh

专题命中 视觉问答 ：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Med-Flamingo: a Multimodal Medical Few-shot Learner

Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, Jure Leskovec

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Preprint

Is Multimodal Vision Supervision Beneficial to Language?

Avinash Madasu, Vasudev Lal

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory

Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, Alireza Fathi

专题命中 视觉问答 ：visual language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Published on CVPR 2023

Towards Models that Can See and Read

Roy Ganz, Oren Nuriel, Aviad Aberdam, Yair Kittenplon, Shai Mazor, Ron Litman

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

PaLM-E: An Embodied Multimodal Language Model

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.AI、cs.LG

Compound Tokens: Channel Fusion for Vision-Language Representation Learning

Maxwell Mbabilla Aladago, AJ Piergiovanni

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, Jianlong Fu

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

12-in-1: Multi-Task Vision and Language Representation Learning

Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, Stefan Lee

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments Jiasen Lu and Vedanuj Goswami contributed equally to this work

Modulated Self-attention Convolutional Network for VQA

Jean-Benoit Delbrouck, Antoine Maiorca, Nathan Hubens, Stéphane Dupont

专题命中 视觉问答 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG

Comments Accepted at NeurIPS 2019 workshop: ViGIL

Visual Coreference Resolution in Visual Dialog using Neural Module Networks

Satwik Kottur, José M. F. Moura, Devi Parikh, Dhruv Batra, Marcus Rohrbach

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments ECCV 2018 + results on VisDial v1.0 dataset

Visual7W: Grounded Question Answering in Images

Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei

专题命中 视觉问答 ：visual question answering(abstract)；grounding(abstract)；分类 cs.CV、cs.LG

Comments CVPR 2016

Explicit Evidence Grounding via Structured Inline Citation Generation

通过结构化内联引文生成实现显式证据基础

Anar Yeginbergen, Amelie Wührl, Anna Rogers, Rodrigo Agerri

机构 * University of the Basque Country (UPV/EHU)（巴斯克大学） ； IT University of Copenhagen（哥本哈根IT大学） 纠错

专题命中 视觉问答 ：grounding(title)

AI总结 提出FullCite框架，通过提示生成、约束解码和后处理跨度对齐三种策略生成结构化内联引文，在三个QA基准上评估引文质量和忠实性，发现LLMs虽能识别相关文档但难以精确定位支持性证据跨度。