视觉大模型 / VLM

Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models

Bowei Tian, Xuntao Lyu, Meng Liu, Hongyi Wang, Ang Li

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.AI、cs.LG

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICLR 2025 Oral

AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models

Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Yunhao Chen, Jitao Sang, Dit-Yan Yeung

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR 2025

Hyperbolic Safety-Aware Vision-Language Models

Tobia Poppi, Tejaswi Kasarla, Pascal Mettes, Lorenzo Baraldi, Rita Cucchiara

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

The in-context inductive biases of vision-language models differ across modalities

Kelsey Allen, Ishita Dasgupta, Eliza Kosoy, Andrew K. Lampinen

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments 11 pages

TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction

Chao Wang, Weiwei Fu, Yang Zhou

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models

Shicheng Xu, Liang Pang, Yunchang Zhu, Huawei Shen, Xueqi Cheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025

CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models

Zihao Sheng, Zilin Huang, Yansong Qu, Yue Leng, Sruthi Bhavanam, Sikai Chen

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptation

Yuheng Ji, Yue Liu, Zhicheng Zhang, Zhao Zhang, Yuting Zhao, Xiaoshuai Hao, Gang Zhou, Xingwei Zhang, Xiaolong Zheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding

Kyungmin Min, Minbeom Kim, Kang-il Lee, Dongryeol Lee, Kyomin Jung

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments NAACL 2025 (Findings); Renamed SGD to SumGD in Summary-Guided Decoding to prevent confusion with Stochastic Gradient Descent

Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models

Xu Han, Linghao Jin, Xuezhe Ma, Xiaofeng Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Models

Sangmin Woo, Jaehyuk Jang, Donguk Kim, Yubin Choi, Changick Kim

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments Project: https://sangminwoo.github.io/RITUAL/

Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models

Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Under review. Project pages: https://github.com/YiLunLee/VCD_Analysis

Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship Classification

Long Lan, Fengxiang Wang, Xiangtao Zheng, Zengmao Wang, Xinwang Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments It has been accepted by TGRS

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Yuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen Kan

专题命中 幻觉与鲁棒性 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective

Yanan Zhang, Jiangmeng Li, Lixiang Liu, Wenwen Qiang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted by NeurIPS 2024

Identifying Implicit Social Biases in Vision-Language Models

Kimia Hamidieh, Haoran Zhang, Walter Gerych, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models

Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, Wenhao Zheng, Zhaoyang Wang, Xiao Wang, Xuchao Zhang, Chetan Bansal, Marc Niethammer, Junzhou Huang, Hongtu Zhu, Yun Li, Jimeng Sun, Zongyuan Ge, Gang Li, James Zou, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2024 Datasets and Benchmarks Track

Multi-Object Hallucination in Vision-Language Models

Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F. Fouhey, Joyce Chai

专题命中 幻觉与鲁棒性 ：vision-language model(title)；vision language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024 | Project page: https://multi-object-hallucination.github.io/

Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models

Lu Yu, Haiyang Zhang, Changsheng Xu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024

A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models

Yuhan Liang, Yijun Li, Yumeng Niu, Qianhe Shen, Hangyu Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

White-box Multimodal Jailbreaks Against Large Vision-Language Models

Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, Yu-Gang Jiang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions

Yudong Zhang, Ruobing Xie, Jiansheng Chen, Xingwu Sun, Yu Wang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACM Multimedia 2024 BNI track (Oral)

Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors

Jiachen Sun, Changsheng Wang, Jiongxiao Wang, Yiwei Zhang, Chaowei Xiao

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages

Adversarial Prompt Tuning for Vision-Language Models

Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, Jitao Sang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024

Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models

Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, Bing Qin

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACL 2024 Main Conference. 21 pages, 20 figures

Mitigating Multilingual Hallucination in Large Vision-Language Models

Xiaoye Qu, Mingyang Song, Wei Wei, Jianfeng Dong, Yu Cheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Vision-Language Models under Cultural and Inclusive Considerations

Antonia Karamolegkou, Phillip Rust, Yong Cao, Ruixiang Cui, Anders Søgaard, Daniel Hershcovich

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments HuCLLM @ ACL 2024

MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context

Zishan Gu, Changchang Yin, Fenglin Liu, Ping Zhang

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

An Empirical Study Into What Matters for Calibrating Vision-Language Models

Weijie Tu, Weijian Deng, Dylan Campbell, Stephen Gould, Tom Gedeon

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICML 2024 Camera Ready