视觉大模型 / VLM - arXivDaily 专题

2409.00301 2024-09-04 cs.CV 79%

ContextVLM: Zero-Shot and Few-Shot Context Understanding for Autonomous Driving using Vision Language Models

Shounak Sural, Naren, Ragunathan Rajkumar

专题命中幻觉与鲁棒性：vision language model(title)；vision-language model(abstract)；分类 cs.CV

Comments Accepted at the 27th IEEE International Conference on Intelligent Transportation Systems (ITSC) 2024

2407.07403 2024-07-15 cs.CV 79%

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, Wei Hu

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2407.05342 2024-07-09 cs.CV 79%

Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models

Longxiang Tang, Zhuotao Tian, Kai Li, Chunming He, Hantao Zhou, Hengshuang Zhao, Xiu Li, Jiaya Jia

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments ECCV 2024

2403.14760 2024-07-04 cs.CV 79%

Can 3D Vision-Language Models Truly Understand Natural Language?

Weipeng Deng, Jihan Yang, Runyu Ding, Jiahui Liu, Yijiang Li, Xiaojuan Qi, Edith Ngai

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments https://github.com/VincentDENGP/3D-LR

2406.04031 2024-07-02 cs.CV cs.CR 79%

Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt

Zonghao Ying, Aishan Liu, Tianyuan Zhang, Zhengmin Yu, Siyuan Liang, Xianglong Liu, Dacheng Tao

专题命中幻觉与鲁棒性：vision language model(title,abstract)；分类 cs.CV

2311.17600 2024-06-21 cs.CV 79%

MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models

Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, Yu Qiao

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2406.07594 2024-06-18 cs.CL cs.AI cs.CR 79%

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Tianle Gu, Zeyang Zhou, Kexin Huang, Dandan Liang, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Xingge Qiao, Keqing Wang, Yujiu Yang, Yan Teng, Yu Qiao, Yingchun Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

2406.09781 2024-06-17 cs.CV 79%

GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding

Yiqi Wu, Xiaodan Hu, Ziming Fu, Siling Zhou, Jiangong Li

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2403.03627 2024-04-29 cs.CL cs.AI 79%

Multimodal Large Language Models to Support Real-World Fact-Checking

Jiahui Geng, Yova Kementchedjhieva, Preslav Nakov, Iryna Gurevych

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

2402.07329 2024-04-17 cs.CV 79%

The Bias of Harmful Label Associations in Vision-Language Models

Caner Hazirbas, Alicia Sun, Yonathan Efroni, Mark Ibrahim

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2310.14566 2024-03-26 cs.CV cs.CL 79%

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, Tianyi Zhou

专题命中幻觉与鲁棒性：vision-language model(title)；LLaVA(abstract)；分类 cs.CV

Comments Accepted to CVPR 2024

2401.11170 2024-03-25 cs.CV cs.CR 79%

Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images

Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, Wei Liu

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ICLR 2024

2401.07567 2024-01-22 cs.CV 79%

Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video

Zhaobo Qi, Yibo Yuan, Xiaowen Ruan, Shuhui Wang, Weigang Zhang, Qingming Huang

专题命中幻觉与鲁棒性：grounding(title,abstract)；分类 cs.CV

Comments accepted by AAAI 2024

2311.09024 2024-01-05 cs.CV 79%

Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing

A K Nirala, A Joshi, C Hegde, S Sarkar

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2312.10665 2023-12-19 cs.CV cs.CL 79%

Silkie: Preference Distillation for Large Visual Language Models

Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong

专题命中幻觉与鲁棒性：visual language model(title)；vision language model(abstract)；分类 cs.CV

Comments Project page: https://vlf-silkie.github.io

2312.01701 2023-12-05 cs.CV cs.CL 79%

Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites

Lei Wang, Jiabang He, Shenshen Li, Ning Liu, Ee-Peng Lim

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments MMM 2024

2305.10355 2023-10-27 cs.CV cs.CL cs.MM 79%

Evaluating Object Hallucination in Large Vision-Language Models

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, Ji-Rong Wen

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to EMNLP 2023

2308.10061 2023-08-22 cs.CV 79%

DPL: Decoupled Prompt Learning for Vision-Language Models

Chen Xu, Yuhan Zhu, Guozhen Zhang, Haocheng Shen, Yixuan Liao, Xiaoxin Chen, Gangshan Wu, Limin Wang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments 11 pages, 5 figures, 8 tables

2304.08479 2023-04-18 cs.CV 79%

Towards Robust Prompts on Vision-Language Models

Jindong Gu, Ahmad Beirami, Xuezhi Wang, Alex Beutel, Philip Torr, Yao Qin

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2303.10431 2023-03-21 cs.CV 79%

DeAR: Debiasing Vision-Language Models with Additive Residuals

Ashish Seth, Mayur Hemani, Chirag Agarwal

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to CVPR'23. Codes and dataset will be released soon

1712.02051 2018-05-23 cs.CV 79%

Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning

Hongge Chen, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Cho-Jui Hsieh

专题命中幻觉与鲁棒性：grounding(title,abstract)；分类 cs.CV

Comments Accepted by 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018). Hongge Chen and Huan Zhang contribute equally to this work

2601.03100 2026-08-11 cs.CV cs.AI 版本更新 79%

TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs

基于文本引导的层融合缓解多模态大语言模型中的幻觉

Chenchen Lin, Sanbao Su, Rachel Luo, Yuxiao Chen, Yan Wang, Marco Pavone, Fei Miao

机构 * University of Connecticut（康涅狄格大学）； NVIDIA（NVIDIA公司）； Stanford University（斯坦福大学）

专题命中幻觉与鲁棒性：LLaVA(abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结 TGIF通过文本引导的层融合方法，有效缓解多模态大语言模型中的幻觉问题，提升视觉接地能力。

Comments Accepted at COLM 2026

2608.03691 2026-08-05 cs.SE cs.AI cs.CV 新提交 79%

Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation

像素上的模式：测量多模态代码生成中的模式完成偏差

Khai-Nguyen Nguyen, Oscar Chaparro, Antonio Mastropaolo

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结该研究针对多模态代码生成中存在的模式完成偏差问题，构建了首个视觉模式完成偏差基准，评估发现前沿MLLMs均存在严重偏差，且偏差与视觉显著性密切相关。

Comments 41st IEEE/ACM International Conference on Automated Software Engineering

2607.26042 2026-07-29 cs.CV cs.LG 新提交 79%

VetClaw: An Edge-Cloud Multimodal Agentic System for Veterinary Disease Screening

VetClaw：一种用于兽医疾病筛查的边缘云多模态智能系统

Syed Mhamudul Hasan, Anas AlSobeh, Hussein Zangoti, Abdur R. Shahid

机构 * Southern Illinois University（南伊利诺伊大学）； Utah Valley University（犹他谷大学）； Jazan University（吉赞大学）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.LG

AI总结 VetClaw是用于兽医疾病筛查的边缘云多模态智能系统，用相机模块采集图像等并发送至视觉语言模型分类。它分离智能体交互与工作流编排，超越静态图像分类，症状引导和多模态输入提升性能，将静态模型转变为多功能安全感知系统。

2508.05502 2026-07-24 cs.CV cs.AI cs.CL 版本更新 79%

MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs

MELLA：弥合低资源语言多模态大语言模型的语言能力与文化根基

Yufei Gao, Jiaying Fei, Nuo Chen, Ruirui Chen, Guohang Yan, Yunshi Lan, Botian Shi

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）； East China Normal University（东华大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Institute of High Performance Computing, A*STAR（高性能计算研究所，A*STAR）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结研究针对低资源语言MLLMs文化内涵不足问题，提出MELLA数据集，采用双源策略，结合母语网络图像-替代文本对与生成翻译的图像描述进行监督，经实验表明能减轻文化幻觉，强调数据对齐对低资源语言文化基础多模态理解的重要性。

2603.03989 2026-07-23 cs.CV cs.AI 版本更新 79%

When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models

当视觉证据模糊时： pareidolia 作为视觉模型的诊断探针

Qianpu Chen, Derya Soydaner, Rob Saunders

机构 * Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands（莱顿高级计算机科学研究所（LIACS）、莱顿大学、莱顿、荷兰）

专题命中幻觉与鲁棒性：LLaVA(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结本文研究了视觉模型在模糊证据下的行为，通过分析pareidolia现象揭示了不同模型在表示层面的差异，发现语义过激活和不确定性策略等机制，为提升视觉语言系统的语义鲁棒性提供诊断和改进方法。

2607.16247 2026-07-21 cs.LG cs.CV 新提交 79%

Self-Evolving Just-In-Time Memory for Proactive Embodied Safety

用于主动具身安全的自进化即时记忆

Bingrui Sima, Lizhong Wang, Xiaoya Lu, Kun He, Xiao Yang

机构 * Huazhong University of Science and Technology（华中科技大学）； Tsinghua University（清华大学）； Shanghai Jiao Tong University（上海交通大学）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.LG

AI总结研究视觉语言模型在闭环交互中应对动态危险的问题，提出自进化即时记忆框架，含RSG、事实记忆和经验记忆，并通过自动测试-验证-写入循环完善元技能，实验证明该框架大幅提升安全成功率且不阻碍任务进展。

2605.18419 2026-07-08 cs.CV cs.AI 版本更新 79%

Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology

面向几何的不确定性聚类用于病理学中鲁棒的视觉上下文学习

Franciskus Xaverius Erick, Johanna Paula Müller, Bernhard Kainz

机构 * FAU Erlangen-Nürnberg, Erlangen, DE（埃尔兰根-纽伦堡大学）； Department of Computing, Imperial College London, London, UK（伦敦帝国理工学院计算机系）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结本文提出GAUC，一种无需训练的聚类选择方法，直接在预训练的多模态嵌入空间中操作，通过优化三个目标提升视觉上下文学习的鲁棒性、准确性和校准性。

2607.01897 2026-07-03 cs.LG cs.AI 新提交 79%

Rank-Then-Act: Reward-Free Control from Frame-Order Progress

先排序后行动：基于帧序进展的无奖励控制

Yuriy Maksyuta, George Bredis, Ruslan Rakhimov, Daniil Gavrilov

机构 * T-Tech

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.AI、cs.LG

AI总结提出Rank-Then-Act框架，利用视觉语言模型从专家视频中学习进度排序器，通过斯皮尔曼等级相关奖励函数实现无环境奖励的策略学习，在离散和连续控制任务中达到或超越现有方法。

Comments 20 pages, 15 figures

2606.20770 2026-06-23 cs.CL cs.AI cs.LG 新提交 79%

Beyond 'One Language, One Script': Quantifying Orthographic Bias in Multilingual VLMs with PuMVR

超越“一种语言，一种文字”：用PuMVR量化多语言视觉语言模型中的正字法偏差

Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina

机构 * RediMinds Inc.（RediMinds公司）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract_cn)；visual reasoning(abstract)；分类 cs.AI、cs.LG

AI总结提出PuMVR基准，通过旁遮普语三种文字的图像推理任务，量化多语言视觉语言模型中的文字依赖偏差，发现文字一致性率低至24.8%，视觉输入无法消除偏差。

Comments 22 pages, 4 figures. Accepted to the 4th Workshop on Cross-Cultural Considerations in NLP (C3NLP) @ ACL 2026

1. 幻觉与鲁棒性 2260 篇

ContextVLM: Zero-Shot and Few-Shot Context Understanding for Autonomous Driving using Vision Language Models

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models

Can 3D Vision-Language Models Truly Understand Natural Language?

Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt

MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding

Multimodal Large Language Models to Support Real-World Fact-Checking

The Bias of Harmful Label Associations in Vision-Language Models

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images

Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video

Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing

Silkie: Preference Distillation for Large Visual Language Models

Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites

Evaluating Object Hallucination in Large Vision-Language Models

DPL: Decoupled Prompt Learning for Vision-Language Models

Towards Robust Prompts on Vision-Language Models

DeAR: Debiasing Vision-Language Models with Additive Residuals

Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning

TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs

Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation

VetClaw: An Edge-Cloud Multimodal Agentic System for Veterinary Disease Screening

MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs

When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models

Self-Evolving Just-In-Time Memory for Proactive Embodied Safety

Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology

Rank-Then-Act: Reward-Free Control from Frame-Order Progress

Beyond 'One Language, One Script': Quantifying Orthographic Bias in Multilingual VLMs with PuMVR