视觉大模型 / VLM

Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models

Yue Li, Xin Yi, Dongsheng Shi, Gerard de Melo, Xiaoling Wang, Linlin Wang

机构 * East China Normal University（华东师范大学） ； Hasso Plattner Institute/University of Potsdam（哈索普劳特纳研究所/波茨坦大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ACL 2025 Findings

PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection

Mahdiyar Molahasani, Azadeh Motamedi, Michael Greenspan, Il-Min Kim, Ali Etemad

机构 * Queen’s University（皇后大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted to ICCV 2025

Robust Multimodal Large Language Models Against Modality Conflict

Zongmeng Zhang, Wengang Zhou, Jie Zhao, Houqiang Li

机构 * School of Artificial Intelligence（人工智能学院） ； Data Science, University of Science（数据科学，科学大学） ； Department of Electronic Engineering（电子工程系） ； Information Science, University of Science（信息科学，科学大学） ； Huawei Technologies Co., Ltd.（华为技术有限公司） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

Comments ICML 2025

SEAL: Vision-Language Model-Based Safe End-to-End Cooperative Autonomous Driving with Adaptive Long-Tail Modeling

Junwei You, Pei Li, Zhuoyu Jiang, Zilin Huang, Rui Gan, Haotian Shi, Bin Ran

机构 * Department of Civil and Environmental Engineering, University of Wisconsin–Madison（威斯康星大学麦迪逊分校土木与环境工程系） ； College of Computing and Data Science, Nanyang Technological University（南洋理工大学 computing and Data Science 学院） ； College of Transportation, Tongji University（同济大学交通运输学院） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

On the Domain Robustness of Contrastive Vision-Language Models

Mario Koddenbrock, Rudolf Hoffmann, David Brodmann, Erik Rodner

机构 * KI-Werkstatt/Fachbereich 2, University of Applied Sciences Berlin（柏林应用技术大学人工智能工作室/第二学系） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Deepbench is available at https://github.com/ml-lab-htw/deepbench

CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention

Zekai Ye, Qiming Li, Xiaocheng Feng, Libo Qin, Yichong Huang, Baohang Li, Kui Jiang, Yang Xiang, Zhirui Zhang, Yunfei Lu, Duyu Tang, Dandan Tu, Bing Qin

机构 * Harbin Institute of Technology（哈尔滨工业大学） ； Peng Cheng Laboratory（鹏城实验室） ； Central South University（中南大学） ； Huawei Technologies Co., Ltd（华为技术有限公司） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments ACL2025 Main

SafeCoT: Improving VLM Safety with Minimal Reasoning

Jiachen Ma, Zhanhui Zhou, Chao Yang, Chaochao Lu

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Fudan University（复旦大学） 纠错

专题命中 幻觉与鲁棒性 ：VLM(title)；vision-language model(abstract)；分类 cs.AI、cs.LG

Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models

Xinlong Chen, Yuanxing Zhang, Qiang Liu, Junfei Wu, Fuzheng Zhang, Tieniu Tan

机构 * New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences (CASIA)（模式识别新技术实验室，自动化研究所，中国科学院） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Kuaishou Technology（快手科技） ； Nanjing University（南京大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to Findings of ACL 2025

Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

Liangliang You, Junchi Yao, Shu Yang, Guimin Hu, Lijie Hu, Di Wang

机构 * Provable Responsible AI and Data Analytics (PRADA) Lab（可证明负责任的人工智能与数据分析实验室） ； King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹大学） ； University of Electronic Science and Technology of China（中国电子科学技术大学） ； University of Copenhagen（哥本哈根大学） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

FREE: Fast and Robust Vision Language Models with Early Exits

Divya Jyoti Bajpai, Manjesh Kumar Hanawal

机构 * Department of IEOR, IIT Bombay（工业工程与运营研究系，印度理工学院班加罗尔） 纠错

专题命中 幻觉与鲁棒性 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.LG

Comments To appear at the Association of Computational Linguistics (ACL) 2025 Conference

Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective

Zhaotian Weng, Zijun Gao, Jerone Andrews, Jieyu Zhao

机构 * University of Southern California（南加州大学） ； Sony AI（索尼人工智能） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models

Sangmin Woo, Donguk Kim, Jaehyuk Jang, Yubin Choi, Changick Kim

机构 * KAIST（韩国科学技术院） 纠错

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments ACL 2025 Findings; Project: https://sangminwoo.github.io/AvisC/

BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models

Huu-Thien Tran, Thanh-Dat Truong, Khoa Luu

机构 * CVIU Lab, University of Arkansas（CVIU实验室，阿肯色大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments CVPRW 2025, 8 pages, 4 figures

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models

Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, Xuming Hu

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； Guangxi Zhuang Autonomous Region Big Data Research Institute（广西壮族自治区大数据研究院） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted by Findings of ACL 2025

Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing

Weixing Wang, Zifeng Ding, Jindong Gu, Rui Cao, Christoph Meinel, Gerard de Melo, Haojin Yang

机构 * Hasso Plattner Institute（霍普夫纳研究所） ； University of Potsdam（波茨坦大学） ； University of Cambridge（剑桥大学） ； University of Oxford（牛津大学） ； German University of Digital Science（德国数字科学大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models

Hao Yin, Guangzong Si, Zilei Wang

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation

Zherui Zhang, Jiaxin Wu, Changwei Wang, Rongtao Xu, Longzhao Huang, Wenhao Xu, Wenbo Xu, Li Guo, Shibiao Xu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

Yongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao, Rui Zheng, Jinlan Fu, Zhenfei Yin, Senjie Jin, Yu Qiao, Xuanjing Huang, Feng Zhao, Tao Gui, Jing Shao

机构 * University of Science and Technology of China（中国科学技术大学） ； Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Are vision language models robust to uncertain inputs?

Xi Wang, Eric Nalisnick

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems

Yuhang Wang, Hao Zhou

机构 * Department of Civil & Environmental Engineering , University of South Florida（土木与环境工程系，佛罗里达州立大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title)；VLM(abstract)；分类 cs.CV、cs.LG

Transferable Adversarial Attacks on Black-Box Vision-Language Models

Kai Hu, Weichen Yu, Li Zhang, Alexander Robey, Andy Zou, Chengming Xu, Haoqi Hu, Matt Fredrikson

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title)；visual question answering(abstract)；分类 cs.CV、cs.LG

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models

Sangmin Woo, Kang Zhou, Yun Zhou, Shuai Wang, Sheng Guan, Haibo Ding, Lin Lee Cheong

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments NAACL 2025

Owls are wise and foxes are unfaithful: Uncovering animal stereotypes in vision-language models

Tabinda Aman, Mohammad Nadeem, Shahab Saquib Sohail, Mohammad Anas, Erik Cambria

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation

Yunpu Zhao, Rui Zhang, Junbin Xiao, Ruibo Hou, Jiaming Guo, Zihao Zhang, Yifan Hao, Yunji Chen

机构 * University of Science and Technology of China（中国科学技术大学） ； SKL of Processors, Institute of Computing Technology, CAS（中国科学院计算技术研究所处理器专项实验室） ； National University of Singapore（新加坡国立大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models

Chung-En, Yu, Hsuan-Chih, Chen, Brian Jalaian, Nathaniel D. Bastian

机构 * University of West Florida（西弗吉尼亚大学） ； New York University（纽约大学） ； United States Military Academy（美国军事学院） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models

Simon Schrodi, David T. Hoffmann, Max Argus, Volker Fischer, Thomas Brox

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICLR 2025 (Oral)

Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models

Zhaochen Wang, Bryan Hooi, Yiwei Wang, Ming-Hsuan Yang, Zi Huang, Yujun Cai

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Under review at COLM 2025

SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments

Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, Qing Guo

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance

Qingpei Guo, Kaiyou Song, Zipeng Feng, Ziping Ma, Qinglong Zhang, Sirui Gao, Xuzheng Yu, Yunxiao Sun, Tai-Wei Chang, Jingdong Chen, Ming Yang, Jun Zhou

专题命中 幻觉与鲁棒性 ：MLLM(title,abstract)；分类 cs.AI、cs.LG

Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models

Mirko Borszukovszki, Ivo Pascal de Jong, Matias Valdenegro-Toro

专题命中 幻觉与鲁棒性 ：vision-language model(title)；visual language model(abstract)；分类 cs.CV、cs.LG

Comments 10 pages, 11 figures, TrustNLP Workshop @ NAACL 2025 Camera ready