视觉大模型 / VLM - arXivDaily 专题

2405.05256 2025-04-04 cs.CV cs.AI cs.LG 82%

THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models

Prannay Kaul, Zhizhong Li, Hao Yang, Yonatan Dukler, Ashwin Swaminathan, C. J. Taylor, Stefano Soatto

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments In CVPR 2024. Code https://github.com/amazon-science/THRONE

2412.02946 2025-03-31 cs.CV cs.AI cs.LG cs.MM 82%

Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis

Po-Hsuan Huang, Jeng-Lin Li, Chin-Po Chen, Ming-Ching Chang, Wei-Chao Chen

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by WACV2025

Journal ref https://openaccess.thecvf.com/content/WACV2025/papers/Huang_Who_Brings_the_Frisbee_Probing_Hidden_Hallucination_Factors_in_Large_WACV_2025_paper.pdf

2503.02199 2025-03-05 cs.CV cs.AI cs.CL cs.LG cs.MM 82%

Words or Vision: Do Vision-Language Models Have Blind Faith in Text?

Ailin Deng, Tri Cao, Zhirui Chen, Bryan Hooi

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2025

2410.11779 2025-02-25 cs.CL cs.AI cs.CV cs.LG cs.MM 82%

MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation

Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, Huajun Chen

专题命中幻觉与鲁棒性：MLLM(title)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2025

2406.11230 2025-02-12 cs.LG cs.AI cs.CL cs.CV 82%

Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models

Hengyi Wang, Haizhou Shi, Shiwei Tan, Weiyi Qin, Wenyuan Wang, Tunyu Zhang, Akshay Nambi, Tanuja Ganu, Hao Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at NAACL 2025 Main

2502.00653 2025-02-04 cs.CR 82%

Towards Robust Multimodal Large Language Models Against Jailbreak Attacks

Ziyi Yin, Yuanpu Cao, Han Liu, Ting Wang, Jinghui Chen, Fenhlong Ma

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；MLLM(abstract)

2501.16378 2025-01-29 cs.LG cs.AI cs.CL cs.CV 82%

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Qing Li, Jiahui Geng, Zongxiong Chen, Kun Song, Lei Ma, Fakhri Karray

专题命中幻觉与鲁棒性：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2501.15046 2025-01-28 cs.CV cs.AI cs.LG 82%

Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities

Shounak Datta, Dhanasekar Sundararaman

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2501.10057 2025-01-20 cs.CL 82%

MSTS: A Multimodal Safety Test Suite for Vision-Language Models

Paul Röttger, Giuseppe Attanasio, Felix Friedrich, Janis Goldzycher, Alicia Parrish, Rishabh Bhardwaj, Chiara Di Bonaventura, Roman Eng, Gaia El Khoury Geagea, Sujata Goswami, Jieun Han, Dirk Hovy, Seogyeong Jeong, Paloma Jeretič, Flor Miriam Plaza-del-Arco, Donya Rooein, Patrick Schramowski, Anastassia Shaitarova, Xudong Shen, Richard Willats, Andrea Zugarini, Bertie Vidgen

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；VLM(abstract)

Comments under review

2501.02029 2025-01-07 cs.LG cs.AI cs.CR cs.CV 82%

Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models

Ziwei Zheng, Junyao Zhao, Le Yang, Lijun He, Fan Li

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2404.14233 2025-01-07 cs.CV cs.AI cs.CL cs.LG 82%

Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Fangxun Shu, Hao Jiang, Linchao Zhu

专题命中幻觉与鲁棒性：vision language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments repo: https://github.com/Mr-Loevan/HSA-DPO

2310.16045 2024-12-12 cs.CV cs.AI cs.CL cs.LG 82%

Woodpecker: Hallucination Correction for Multimodal Large Language Models

Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, Enhong Chen

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by Science China Information Sciences (SCIS)

Journal ref SCIENCE CHINA Information Sciences, 2024

2406.07057 2024-12-09 cs.CL cs.AI cs.CV cs.LG 82%

MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models

Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang, Huanran Chen, Xiao Yang, Xingxing Wei, Hang Su, Yinpeng Dong, Jun Zhu

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 100 pages, 84 figures, 33 tables

2411.17760 2024-11-28 cs.CL cs.AI cs.CV cs.LG 82%

Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach

Shijian Deng, Wentian Zhao, Yu-Jhe Li, Kun Wan, Daniel Miranda, Ajinkya Kale, Yapeng Tian

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2407.21001 2024-10-28 cs.CV cs.AI cs.LG 82%

GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models

Ali Abdollahi, Mahdi Ghaznavi, Mohammad Reza Karimi Nejad, Arash Mari Oriyad, Reza Abbasi, Ali Salesi, Melika Behjati, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref Volume 392 of ECAI 2024, Pages 729 - 736

2410.18927 2024-10-25 cs.CR 82%

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Zonghao Ying, Aishan Liu, Siyuan Liang, Lei Huang, Jinyang Guo, Wenbo Zhou, Xianglong Liu, Dacheng Tao

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；MLLM(abstract)

2410.15778 2024-10-23 cs.CV cs.AI cs.LG cs.MM 82%

Reducing Hallucinations in Vision-Language Models via Latent Space Steering

Sheng Liu, Haotian Ye, Lei Xing, James Zou

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 21 pages

2408.08464 2024-10-23 cs.CR 82%

$\textit{MMJ-Bench}$: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models

Fenghua Weng, Yue Xu, Chengyan Fu, Wenjie Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；MLLM(abstract)

2410.08616 2024-10-14 cs.RO 82%

Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking

Wei Zhang, Pengfei Li, Junli Wang, Bingchuan Sun, Qihao Jin, Guangjun Bao, Shibo Rui, Yang Yu, Wenchao Ding, Peng Li, Yilun Chen

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；MLLM(abstract)

2410.06699 2024-10-10 cs.CV cs.AI cs.LG 82%

Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models

Yubo Wang, Chaohu Liu, Yanqiu Qu, Haoyu Cao, Deqiang Jiang, Linli Xu

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ACMMM 2024

2409.11365 2024-10-10 cs.CL 82%

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, Lanqing Hong, Lingpeng Kong, Xin Jiang, Zhenguo Li

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；MLLM(abstract)

Comments 10 pages, COLM-2024

2408.13906 2024-08-27 cs.CV cs.AI cs.LG 82%

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Yeji Park, Deokyeong Lee, Junsuk Choe, Buru Chang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments First two authors contributed equally. Source code is available at https://github.com/yejipark-m/ConVis

2407.11121 2024-07-17 cs.CV cs.AI cs.LG 82%

Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques

Rishika Bhagwatkar, Shravan Nayak, Reza Bayat, Alexis Roger, Daniel Z Kaplan, Pouya Bashivan, Irina Rish

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2402.01345 2024-05-16 cs.CV cs.AI cs.CL cs.LG 82%

Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models

Zongbo Han, Zechen Bai, Haiyang Mei, Qianli Xu, Changqing Zhang, Mike Zheng Shou

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2404.16804 2024-04-26 cs.CV cs.AI cs.LG 82%

AAPL: Adding Attributes to Prompt Learning for Vision-Language Models

Gahyeon Kim, Sohee Kim, Seokju Lee

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2024 Workshop on Prompting in Vision, Project Page: https://github.com/Gahyeonkim09/AAPL

2403.14119 2024-04-02 cs.CV cs.AI cs.CL cs.LG 82%

C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion

Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark Hasegawa-Johnson, Yingzhen Li, Chang D. Yoo

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2024

2403.01849 2024-03-05 cs.CV cs.AI cs.LG 82%

One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models

Lin Li, Haoyan Guan, Jianing Qiu, Michael Spratling

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2024

2308.15126 2023-10-11 cs.LG cs.AI cs.CL cs.CV 82%

Evaluation and Analysis of Hallucination in Large Vision-Language Models

Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, Jitao Sang, Haoyu Tang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 11 pages, 5 figures

2307.11978 2023-07-25 cs.CV cs.AI cs.LG 82%

Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?

Cheng-En Wu, Yu Tian, Haichao Yu, Heng Wang, Pedro Morgado, Yu Hen Hu, Linjie Yang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICCV2023

2606.03793 2026-08-12 cs.CL cs.CV 版本更新 81%

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

探索多语言多模态大语言模型的对抗鲁棒性与安全对齐

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

机构 * Mohamed Bin Zayed University of AI, UAE（穆罕默德·本·扎耶德人工智能大学，阿联酋）； Khalifa University, UAE（卡比拉大学，阿联酋）； Australian National University, Australia（澳大利亚国立大学，澳大利亚）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

AI总结本研究通过梯度攻击和跨语言评估，发现多语言多模态大语言模型存在可迁移的对抗脆弱性，并揭示低资源语言因理解失败而呈现的虚假安全现象，提出深层训练整合才能实现真正的多语言安全对齐。

Journal ref The 37th British Machine Vision Conference (BMVC) 2026

1. 幻觉与鲁棒性 2257 篇

THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models

Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis

Words or Vision: Do Vision-Language Models Have Blind Faith in Text?

MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation

Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models

Towards Robust Multimodal Large Language Models Against Jailbreak Attacks

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities

MSTS: A Multimodal Safety Test Suite for Vision-Language Models

Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models

Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

Woodpecker: Hallucination Correction for Multimodal Large Language Models

MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models

Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach

GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Reducing Hallucinations in Vision-Language Models via Latent Space Steering

$\textit{MMJ-Bench}$: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models

Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking

Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques

Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models

AAPL: Adding Attributes to Prompt Learning for Vision-Language Models

C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion

One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models

Evaluation and Analysis of Hallucination in Large Vision-Language Models

Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models