视觉大模型 / VLM - arXivDaily 专题

2509.08777 2025-09-11 cs.CV cs.CL 79%

Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles

Eric Slyman, Mehrab Tanjim, Kushal Kafle, Stefan Lee

机构 * Adobe（Adobe公司）； Oregon State University（俄勒冈州立大学）

专题命中幻觉与鲁棒性：MLLM(title)；multimodal large language model(abstract)；分类 cs.CV

Comments 17 pages, 8 figures, Accepted at ICCV 2025

2509.06992 2025-09-10 cs.CV 79%

FedAPT: Federated Adversarial Prompt Tuning for Vision-Language Models

Kun Zhai, Siheng Chen, Xingjun Ma, Yu-Gang Jiang

机构 * Fudan University, Shanghai, China（复旦大学）； Shanghai Jiao Tong University, Shanghai, China（上海交通大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments ACM MM25

2508.15370 2025-08-22 cs.CL cs.AI 79%

Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation

Yichi Zhang, Yao Huang, Yifan Wang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Huanran Chen, Xiao Yang, Xingxing Wei, Hang Su, Yinpeng Dong, Jun Zhu

机构 * Department of Computer Science and Technology, College of AI, Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University（计算机科学与技术系、人工智能学院、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学）； Institute of Artificial Intelligence, Beihang University（人工智能研究院、北航）； RealAI

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments For Appendix, please refer to arXiv:2406.07057

2508.12861 2025-08-19 cs.CV 79%

Cross-Domain Few-Shot Learning via Multi-View Collaborative Optimization with Vision-Language Models

Dexia Chen, Wentao Zhang, Qianjie Zhu, Ping Hu, Weibing Li, Tong Zhang, Ruixuan Wang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2412.05934 2025-08-19 cs.CR cs.AI 79%

Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models

Ma Teng, Jia Xiaojun, Duan Ranjie, Li Xinfeng, Huang Yihao, Jia Xiaoshuang, Chu Zhixuan, Ren Wenqi

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments ICCV 2025

2507.09500 2025-08-14 cs.CV 79%

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations

Yiwen Liang, Hui Chen, Yizhe Xiong, Zihan Zhou, Mengyao Lyu, Zijia Lin, Shuaicheng Niu, Sicheng Zhao, Jungong Han, Guiguang Ding

机构 * Tsinghua University（清华大学）； Nanyang Technological University（南洋理工大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at the 33rd ACM International Conference on Multimedia(ACM MM 2025)

2508.05087 2025-08-08 cs.MM cs.AI cs.CL cs.CR 79%

JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering

Renmiao Chen, Shiyao Cui, Xuancheng Huang, Chengwei Pan, Victor Shea-Jay Huang, QingLin Zhang, Xuan Ouyang, Zhexin Zhang, Hongning Wang, Minlie Huang

机构 * CoAI group, DCST, Tsinghua University（清华大学DCST学院）； Beihang University（北航大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments 10 pages, 3 tables, 2 figures, to appear in the Proceedings of the 33rd ACM International Conference on Multimedia (MM '25)

2508.05083 2025-08-08 cs.AI 79%

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

Dexuan Xu, Jieyi Wang, Zhongyan Chai, Yongzhi Cao, Hanpin Wang, Huamin Zhang, Yu Huang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments 18 pages

2508.03469 2025-08-06 cs.CV 79%

IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models

Jiabing Yang, Chenhang Cui, Yiyang Zhou, Yixiang Chen, Peng Xia, Ying Wei, Tao Yu, Yan Huang, Liang Wang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2506.16218 2025-07-31 cs.CV 79%

FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Models

Xinting Liao, Weiming Liu, Jiaming Qian, Pengyang Zhou, Jiahe Xu, Wenjie Wang, Chaochao Chen, Xiaolin Zheng, Tat-Seng Chua

机构 * Zhejiang University（浙江大学）； National University of Singapore（新加坡国立大学）； University of Science and Technology of China（中国科学技术大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ICML25

2507.21637 2025-07-30 cs.AI 79%

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Wanying Wang, Zeyu Ma, Han Zheng, Xin Tan, Mingang Chen

机构 * Shanghai Key Laboratory of Computer Software Testing and Evaluating（上海软件测试与评估 key laboratory）； Shanghai Normal University（上海Normal University）； TrustAI Pte. Ltd. ； East China Normal University（东华师范大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.AI

Comments Accepted by ACM Multimedia 2025

2507.21521 2025-07-30 cs.CV 79%

Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration

Athmanarayanan Lakshmi Narayanan, Amrutha Machireddy, Ranganath Krishnan

机构 * Intel Labs（英特尔实验室）； Intel Corporation（英特尔公司）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments International Joint Conference on Neural Networks 2025 (Accepted)

2507.21494 2025-07-30 cs.LG 79%

Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning

Wenxuan Bao, Ruxi Deng, Ruizhong Qiu, Tianxin Wei, Hanghang Tong, Jingrui He

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.LG

Comments Accepted by ICCV 2025

2507.15652 2025-07-22 cs.CV 79%

Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

Haoran Zhou, Zihan Zhang, Hao Chen

机构 * Southeast University（东南大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2405.20090 2025-07-22 cs.CV 79%

Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models

Hao Cheng, Erjia Xiao, Jiayan Yang, Jinhao Duan, Yichi Wang, Jiahang Cao, Qiang Zhang, Le Yang, Kaidi Xu, Jindong Gu, Renjing Xu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））； University of Oxford（牛津大学）； Drexel University（德雷塞尔大学）； Beijing University of Technology（北京理工大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Xi’an Jiaotong University（西安交通大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments This paper is accepted by ACM MM 2025

2507.12950 2025-07-21 cs.LG 79%

Insights into a radiology-specialised multimodal large language model with sparse autoencoders

Kenza Bouzid, Shruthi Bannur, Felix Meissen, Daniel Coelho de Castro, Anton Schwaighofer, Javier Alvarez-Valle, Stephanie L. Hyland

机构 * Microsoft Research, Health Futures, Cambridge, United Kingdom（微软研究院，健康未来，剑桥，英国）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.LG

Comments Actionable Interpretability Workshop at ICML 2025. 24 pages, 7 figures, 5 tables

2507.07709 2025-07-11 cs.CV 79%

One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models

Jiale Zhao, Xinyang Jiang, Junyao Gao, Yuhao Xue, Cairong Zhao

机构 * Tongji University（同济大学）； Microsoft Research Asia（微软亚洲研究院）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2506.23101 2025-07-08 cs.CL cs.AI cs.CY 79%

From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship

Yue Xu, Wenjie Wang

机构 * School of Information Science and Technology, ShanghaiTech University（信息科学与技术学院，上海科技大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

2409.09318 2025-07-08 cs.CL cs.CV 79%

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

Yahan Tu, Rui Hu, Jitao Sang

机构 * Beijing Jiaotong University（北京交通大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2506.23590 2025-07-01 cs.CV 79%

CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models

Qiming Li, Zekai Ye, Xiaocheng Feng, Weihong Zhong, Libo Qin, Ruihan Chen, Baohang Li, Kui Jiang, Yaowei Wang, Ting Liu, Bing Qin

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2506.22979 2025-07-01 cs.CV 79%

Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation

Jie Liu, Jiayi Shen, Pan Zhou, Jan-Jakob Sonke, Efstratios Gavves

机构 * University of Amsterdam（阿姆斯特丹大学）； Singapore Management University（新加坡管理大学）； The Netherlands Cancer Institute（荷兰癌症研究院）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments ICCV2025 Proceeding

2501.04931 2025-06-30 cs.CR cs.AI cs.CL 79%

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei

机构 * Institution1（机构1）； Institution2（机构2）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments ICCV2025

2506.16760 2025-06-23 cs.CL cs.CV 79%

Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

Lei Jiang, Zixun Zhang, Zizhou Wang, Xiaobing Sun, Zhen Li, Liangli Zhen, Xiaohua Xu

机构 * University of Science and Technology of China（中国科学技术大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments 15 pages, 9 figures

2501.00848 2025-06-23 cs.CV 79%

IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models

Yiming Zhang, Zicheng Zhang, Xinyi Wei, Xiaohong Liu, Guangtao Zhai, Xiongkuo Min

机构 * Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, China（图像通信与网络工程研究院，上海交通大学）

专题命中幻觉与鲁棒性：vision-language model(title)；visual language model(abstract)；分类 cs.CV

2502.10794 2025-06-18 cs.CV 79%

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, Changyu Dong

机构 * Guangzhou University（广州大学）； Shanghai Jiao Tong University（上海交通大学）； Australian Institute for Machine Learning, The University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments CVPR 2025 highlight

2506.06076 2025-06-09 cs.CV 79%

Full Conformal Adaptation of Medical Vision-Language Models

Julio Silva-Rodríguez, Leo Fillioux, Paul-Henry Cournède, Maria Vakalopoulou, Stergios Christodoulidis, Ismail Ben Ayed, Jose Dolz

机构 * ÉTS Montréal（蒙特利尔ÉTS学院）； CentraleSupélec, Université Paris-Saclay（中央理工学院，巴黎萨克雷大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments IPMI 2025. Code: https://github.com/jusiro/FCA

2506.03371 2025-06-05 cs.CV 79%

Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views

Xiaonan Wang, Bo Shao, Hansaem Kim

专题命中幻觉与鲁棒性：VLM(title)；vision-language model(abstract)；分类 cs.CV

2407.19719 2025-06-03 cs.CV 79%

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Jiaxin Zhang, Yunqin Li, Tomohiro Fukuda, Bowen Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments 15 pages, 10 figures

2410.02681 2025-05-30 cs.LG 79%

Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

Shuoyuan Wang, Yixuan Li, Hongxin Wei

机构 * Shuoyuan Wang（王舒元）； Yixuan Li（李奕轩）； Hongxin Wei（魏宏鑫）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.LG

Comments Accepted by ICML 2025

2505.22305 2025-05-29 cs.CV 79%

IKIWISI: An Interactive Visual Pattern Generator for Evaluating the Reliability of Vision-Language Models Without Ground Truth

Md Touhidul Islam, Imran Kabir, Md Alimoor Reza, Syed Masum Billah

机构 * Pennsylvania State University（宾夕法尼亚州立大学）； Drake University（德拉威大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at DIS'25 (Funchal, Portugal)

1. 幻觉与鲁棒性 2260 篇

Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles

FedAPT: Federated Adversarial Prompt Tuning for Vision-Language Models

Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation

Cross-Domain Few-Shot Learning via Multi-View Collaborative Optimization with Vision-Language Models

Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models

Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations

JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models

FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Models

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration

Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning

Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models

Insights into a radiology-specialised multimodal large language model with sparse autoencoders

One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models

From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models

Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

Full Conformal Adaptation of Medical Vision-Language Models

Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

IKIWISI: An Interactive Visual Pattern Generator for Evaluating the Reliability of Vision-Language Models Without Ground Truth