视觉大模型 / VLM

Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography

奥德赛：通过双隐写术对集成多模态大语言模型系统的商业系统进行劫持

Songze Li, Jiameng Cheng, Yiming Li, Xiaojun Jia, Dacheng Tao

机构 * S3IC-Lab（S3IC实验室） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

AI总结 奥德赛通过双隐写术劫持多模态大语言模型集成系统，揭示现有安全过滤器的局限性，实现高达99%的攻击成功率。

Comments This paper is accepted by Network and Distributed System Security Symposium (NDSS) 2026

Adversarial Robustness of Vision in Open Foundation Models

开放基础模型中视觉的对抗鲁棒性

Jonathon Fox, William J Buchanan, Pavlos Papadopoulos

机构 * Blockpass ID Lab（Blockpass ID 实验室） ； Edinburgh Napier University（爱丁堡纳皮尔大学） 纠错

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

AI总结 本文研究了LLaVA-1.5-13B和Llama 3.2 Vision-8B-2在视觉输入下的对抗鲁棒性，发现Llama 3.2 Vision在高扰动水平下性能下降更小，表明视觉模态是降级开放权重VLMs的可行攻击向量。

Journal ref IEEE Access, 2025

Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation

利用策略数据优化LVLMs以实现有效的幻觉缓解

Chengzhi Yu, Yifan Xu, Yifan Chen, Wenyi Zhang

机构 * University of Science and Technology of China（中国科学技术大学） ； Hong Kong Baptist University（香港 Baptist 大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出利用策略数据优化LVLMs，通过幻觉分类器和动态加权DPO算法，显著降低幻觉率并提升模型性能。

Leveraging NTPs for Efficient Hallucination Detection in VLMs

Ofir Azachi, Kfir Eliyahu, Eyal El Ani, Rom Himelstein, Roi Reichart, Yuval Pinter, Nitay Calderon

机构 * Department of Data and Decision Science, Technion - Israel Institute of Technology（数据与决策科学系，技术学院-以色列理工学院） ； Faculty of Computer and Information Science, Ben-Gurion University of the Negev（计算机与信息科学系，贝内-约尔根大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments Accepted to The First Workshop on Confabulation, Hallucinations, & Overgeneration in Multilingual & Precision-critical Setting - AACL-IJCNLP2025

Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs

Liu Yu, Zhonghao Chen, Ping Kuang, Zhikun Feng, Fan Zhou, Lan Wang, Gillian Dobbie

机构 * University of Auckland（奥克兰大学） ； China Scholarship Council（中国留学基金委） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, published to AAAI 2026

ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks

Zhaorun Chen, Xun Liu, Mintong Kang, Jiawei Zhang, Minzhou Pan, Shuang Yang, Bo Li

机构 * University of Chicago（芝加哥大学） ； University of Illinois（伊利诺伊大学） ； Virtue AI ； Meta 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments 60 pages, 16 figures

PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications

Hitesh Laxmichand Patel, Amit Agarwal, Srikant Panda, Hansa Meghwani, Karan Dua, Paul Li, Tao Sheng, Sujith Ravi, Dan Roth

机构 * Oracle AI 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted in EMNLP 2025

CIDER: A Causal Cure for Brand-Obsessed Text-to-Image Models

Fangjian Shen, Zifeng Liang, Chao Wang, Wushao Wen

机构 * School of Computer Science（计算机科学学院） ； Engineering, Sun Yat-sen University, Guangzhou, China（工程学院，中山大学，广州，中国） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 7 figures, submitted to ICASSP2026

Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios

Yunkai Dang, Mengxi Gao, Yibo Yan, Xin Zou, Yanggan Gu, Jungang Li, Jingyu Wang, Peijie Jiang, Aiwei Liu, Jia Liu, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； The Hong Kong University of Science and Technology（香港科技大学） ； Tsinghua University（清华大学） ； Ant Group（蚂蚁集团） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 幻觉与鲁棒性 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.AI、cs.LG

Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring

Zihan Fang, Zheng Lin, Senkang Hu, Yihang Tao, Yiqin Deng, Xianhao Chen, Yuguang Fang

机构 * Hong Kong JC STEM Lab of Smart City and Department of Computer Science, City University of Hong Kong（香港JC STEM实验室及城市大学计算机科学系） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments 14 pages, 10 figures

Grasp-HGN: Grasping the Unexpected

Mehrshad Zandigohar, Mallesham Dasari, Gunar Schirner

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.AI、cs.LG

Comments Paper accepted at ACM Transactions on Embedded Computing Systems

Learning Adaptive Dexterous Grasping from Single Demonstrations

Liangzhi Shi, Yulin Liu, Lingqi Zeng, Bo Ai, Zhengdong Hong, Hao Su

机构 * University of California, San Diego（加州大学圣地亚哥分校） ； Tsinghua University（清华大学） ； Hillbot 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

COEF-VQ: Cost-Efficient Video Quality Understanding through a Cascaded Multimodal LLM Framework

Xin Dong, Sen Jia, Ming Rui Wang, Yan Li, Zhenheng Yang, Bingfeng Deng, Hongyu Xiong

机构 * ByteDance Inc.（字节跳动公司） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration

Ramya Hebbalaguppe, Tamoghno Kandar, Abhinav Nagpal, Chetan Arora

机构 * IIT Delhi（德里印度理工学院） ； TCS Research Labs（塔塔咨询公司研究实验室） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments 26 pages

VLMs Can Aggregate Scattered Training Patches

Zhanhui Zhou, Lingjie Chen, Chao Yang, Chaochao Lu

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding

Yeongjae Cho, Keonwoo Kim, Taebaek Hwang, Sungzoon Cho

机构 * Seoul National University（首尔国立大学） ； Kim & Chang AI&IT System Center（金·昌AI&IT系统中心） ； Waddle Corporation（Waddle公司） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

VLSBench: Unveiling Visual Leakage in Multimodal Safety

Xuhao Hu, Dongrui Liu, Hao Li, Xuanjing Huang, Jing Shao

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Fudan University（复旦大学） ； Beihang University（北京航空航天大学） 纠错

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments ACL2025 Main

SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion

Zihao Pan, Lifeng Chen, Weibin Wu, Yuhang Cao, Zibin Zheng

机构 * School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院） ； Computer Science, Beijing Jiaotong University（北京交通大学计算机科学） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset

Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, Xirong Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2025, Highlight

The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search

Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, David Ha

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

FAA-CLIP: Federated Adversarial Adaptation of CLIP

Yihang Wu, Ahmad Chaddad, Christian Desrosiers, Tareef Daqqaq, Reem Kateb

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments Accepted in IEEE Internet of Things Journal

Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions

Zhijie Tan, Yuzhi Li, Shengwei Meng, Xiang Yuan, Weiping Li, Tong Mo, Bingce Wang, Xu Chu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 2025 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2025)

Is Your Text-to-Image Model Robust to Caption Noise?

Weichen Yu, Ziyan Yang, Shanchuan Lin, Qi Zhao, Jianyi Wang, Liangke Gui, Matt Fredrikson, Lu Jiang

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations

Zhaorun Chen, Francesco Pinto, Minzhou Pan, Bo Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

Comments 43 pages, 20 figures

Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress

Christopher Agia, Rohan Sinha, Jingyun Yang, Zi-ang Cao, Rika Antonova, Marco Pavone, Jeannette Bohg

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://sites.google.com/stanford.edu/sentinel. 35 pages, 9 figures. Accepted to the Conference on Robot Learning (CoRL) 2024

TP-Eval: Tap Multimodal LLMs' Potential in Evaluation by Customizing Prompts

Yuxuan Xie, Tianhua Li, Wenqi Shao, Kaipeng Zhang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

PuzzleAvatar: Assembling 3D Avatars from Personal Albums

Yuliang Xiu, Yufei Ye, Zhen Liu, Dimitrios Tzionas, Michael J. Black

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Page: https://puzzleavatar.is.tue.mpg.de/, Code: https://github.com/YuliangXiu/PuzzleAvatar, Video: https://youtu.be/0hpXH2tVPk4

Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts

Yuanwei Wu, Xiang Li, Yixin Liu, Pan Zhou, Lichao Sun

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges

Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Boosting Cross-task Transferability of Adversarial Patches with Visual Relations

Tony Ma, Songze Li, Yisong Xiao, Shunchang Liu

专题命中 幻觉与鲁棒性 ：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV、cs.LG