多模态大模型

COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptation

Fanding Huang, Jingyan Jiang, Qinting Jiang, Hebei Li, Faisal Nadeem Khan, Zhi Wang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to CVPR 2025

Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis

Po-Hsuan Huang, Jeng-Lin Li, Chin-Po Chen, Ming-Ching Chang, Wei-Chao Chen

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by WACV2025

Journal ref https://openaccess.thecvf.com/content/WACV2025/papers/Huang_Who_Brings_the_Frisbee_Probing_Hidden_Hallucination_Factors_in_Large_WACV_2025_paper.pdf

Squeeze Out Tokens from Sample for Finer-Grained Data Governance

Weixiong Lin, Chen Ju, Haicheng Wang, Shengchao Hu, Shuai Xiao, Mengting Chen, Yuheng Jiao, Mingshuai Yao, Jinsong Lan, Qingwen Liu, Ying Chen

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Small Vision-Language Models: A Survey on Compact Architectures and Techniques

Nitesh Patnaik, Navdeep Nayak, Himani Bansal Agrawal, Moinak Chinmoy Khamaru, Gourav Bal, Saishree Smaranika Panda, Rishi Raj, Vishal Meena, Kartheek Vadlamani

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

On the Limitations of Vision-Language Models in Understanding Image Transforms

Ahmad Mustafa Anis, Hasnain Ali, Saquib Sarfraz

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 8 pages, 15 images

Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention

Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, QianYing Wang, Ping Chen, Xiaoqin Zhang, Shijian Lu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by CVPR 2025

LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression

Souvik Kundu, Anahita Bhiwandiwalla, Sungduk Yu, Phillip Howard, Tiep Le, Sharath Nittur Sridhar, David Cobbley, Hao Kang, Vasudev Lal

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This work has been accepted to NAACL 2025

Advancing vision-language models in front-end development via data synthesis

Tong Ge, Yashu Liu, Jieping Ye, Tianyi Li, Chao Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

VLind-Bench: Measuring Language Priors in Large Vision-Language Models

Kang-il Lee, Minbeom Kim, Seunghyun Yoon, Minsung Kim, Dongryeol Lee, Hyukhun Koh, Kyomin Jung

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NAACL 2025 Findings

Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective

Xiaorui Ma, Haoran Xie, S. Joe Qin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 28 pages, 3 figures

PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding

Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, Yue Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ICLR 2025. Project page: https://physbench.github.io/ Dataset: https://huggingface.co/datasets/USC-GVL/PhysBench

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Qing Li, Jiahui Geng, Zongxiong Chen, Kun Song, Lei Ma, Fakhri Karray

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts

Yuan Zang, Tian Yun, Hao Tan, Trung Bui, Chen Sun

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Transactions on Machine Learning Research, 2025

DriVLM: Domain Adaptation of Vision-Language Models in Autonomous Driving

Xuran Zheng, Chang D. Yoo

专题命中 图文多模态 ：multimodal(abstract)；MLLM(abstract)

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

Shudong Liu, Yiqiao Jin, Cheng Li, Derek F. Wong, Qingsong Wen, Lichao Sun, Haipeng Chen, Xing Xie, Jindong Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical report; 26 pages

Reconstructive Visual Instruction Tuning

Haochen Wang, Anlin Zheng, Yucheng Zhao, Tiancai Wang, Zheng Ge, Xiangyu Zhang, Zhaoxiang Zhang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

Ziqiao Ma, Jiayi Pan, Joyce Chai

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL 2023 Outstanding Paper

MMFactory: A Universal Solution Search Engine for Vision-Language Tasks

Wan-Cyuan Fan, Tanzila Rahman, Leonid Sigal

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Membership Inference Attacks against Large Vision-Language Models

Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, Volkan Cevher

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024

Vision-Language Models Can Self-Improve Reasoning via Reflection

Kanzhi Cheng, Yantao Li, Fangzhi Xu, Jianbing Zhang, Hao Zhou, Yang Liu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

H2OVL-Mississippi Vision Language Models Technical Report

Shaikat Galib, Shanshan Wang, Guanshuo Xu, Pascal Pfeiffer, Ryan Chesler, Mark Landry, Sri Satish Ambati

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts

Anh-Quan Cao, Maximilian Jaritz, Matthieu Guillaumin, Raoul de Charette, Loris Bazzani

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding

Shenghuan Sun, Alexander Schubert, Gregory M. Goldgof, Zhiqing Sun, Thomas Hartvigsen, Atul J. Butte, Ahmed Alaa

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Code available at: https://github.com/AlaaLab/Dr-LLaVA

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, Junyang Lin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Code is available at https://github.com/QwenLM/Qwen2-VL. arXiv admin note: text overlap with arXiv:2408.15262 by other authors

LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation

Mohammad Abuzar Hashemi, Zhanghexuan Li, Mihir Chauhan, Yan Shen, Abhishek Satbhai, Mir Basheer Ali, Mingchen Gao, Sargur Srihari

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 15 pages, 10 Figures, 5 Tables. Accepted for Oral Presentation at Irish Machine Vision and Image Processing Conference Proceedings (IMVIP), 2024

IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning

Soeun Lee, Si-Woo Kim, Taewhan Kim, Dong-Jin Kim

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to EMNLP 2024

Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI

Robert Wolfe, Aayushi Dangol, Alexis Hiniker, Bill Howe

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at Artificial Intelligence, Ethics, and Society 2024

Mitigating Multilingual Hallucination in Large Vision-Language Models

Xiaoye Qu, Mingyang Song, Wei Wei, Jianfeng Dong, Yu Cheng

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Distilling Vision-Language Foundation Models: A Data-Free Approach via Prompt Diversification

Yunyi Xuan, Weijie Chen, Shicai Yang, Di Xie, Luojun Lin, Yueting Zhuang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ACMMM 2023

IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers

Chenglin Yang, Siyuan Qiao, Yuan Cao, Yu Zhang, Tao Zhu, Alan Yuille, Jiahui Yu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments To appear in ECCV 2024