视觉大模型 / VLM - arXivDaily 专题

2504.16104 2025-08-12 cs.CY 78%

Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models

Aditya Gulati, Moreno D'Incà, Nicu Sebe, Bruno Lepri, Nuria Oliver

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)

Comments 39 pages, 4 figures, 33 tables; Accepted for publication at the Eighth AAAI/ACM Conference on AI, Ethics and Society (AIES 2025) (https://www.aies-conference.com/2025/)

2409.09845 2025-08-05 cs.RO 78%

Friction-Aware Safety Locomotion for Wheeled-legged Robots using Vision Language Models and Reinforcement Learning

Bo Peng, Donghoon Baek, Qijie Wang, Joao Ramos

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Department of Electrical and Computer Engineering（电气与计算机工程系）； Department of Mechanical Science Engineering（机械科学与工程系）； School of Software（软件学院）

专题命中幻觉与鲁棒性：vision language model(title)；vision-language model(abstract)

Comments Accepted to Humanoids 2025

2507.10749 2025-07-16 cs.RO 78%

RCG: Safety-Critical Scenario Generation for Robust Autonomous Driving via Real-World Crash Grounding

Benjamin Stoler, Juliet Yang, Jonathan Francis, Jean Oh

机构 * School of Computer Science at Carnegie Mellon University（卡内基梅隆大学计算机科学学院）； Bosch Center for AI（博世人工智能中心）

专题命中幻觉与鲁棒性：grounding(title,abstract)

2502.14744 2025-06-24 cs.CL 78%

HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng, Xiangyu Yue

机构 * MMLab, The Chinese University of Hong Kong（中文大学香港大学）； Future Lab, Alibaba Group（阿里集团未来实验室）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments Accepted by ACL 2025 (Main)

2506.13055 2025-06-17 cs.CL 78%

CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Model

Jiangtong Li, Yiyun Zhu, Dawei Cheng, Zhijun Ding, Changjun Jiang

机构 * School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)

Comments 22 pages, 9 figures

2505.21967 2025-05-29 cs.CL 78%

Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack

Juan Ren, Mark Dras, Usman Naseem

机构 * School of Computing, Macquarie University（计算机学院，麦考瑞大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments Preprint

2503.11619 2025-03-17 cs.CR 78%

Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense

Shuyang Hao, Yiwei Wang, Bryan Hooi, Ming-Hsuan Yang, Jun Liu, Chengcheng Tang, Zi Huang, Yujun Cai

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

2502.20750 2025-03-03 cs.CL 78%

Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow

Jiaqi Bai, Hongcheng Guo, Zhongyuan Peng, Jian Yang, Zhoujun Li, Mohan Li, Zhihong Tian

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments Accepted to AAAI 2025. Camera ready version

2311.14948 2025-01-14 cs.LG cs.AI cs.CV 78%

Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective

Sahil Verma, Gantavya Bhatt, Avi Schwarzschild, Soumye Singhal, Arnav Mohanty Das, Chirag Shah, John P Dickerson, Pin-Yu Chen, Jeff Bilmes

专题命中幻觉与鲁棒性：vision-language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at TMLR (https://openreview.net/forum?id=Conma3qnaT)

2411.10603 2024-11-19 cs.RO cs.SY eess.SY 78%

A Novel MLLM-based Approach for Autonomous Driving in Different Weather Conditions

Sonda Fourati, Wael Jaafar, Noura Baccar

专题命中幻觉与鲁棒性：MLLM(title,abstract)

Comments 9 pages, 6 figures; Submitted to IEEE Transactions on Intelligent Transportation Systems

2410.16977 2024-10-23 cs.CL 78%

IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing

Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, Jufeng Chen

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)

2406.14137 2024-10-21 cs.CL 78%

MACAROON: Training Vision-Language Models To Be Your Engaged Partners

Shujin Wu, Yi R. Fung, Sha Li, Yixin Wan, Kai-Wei Chang, Heng Ji

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments The code will be made public at https://github.com/ShujinWu-0814/MACAROON

2407.21659 2024-10-18 cs.CL 78%

Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models

Yue Xu, Xiuyuan Qi, Zhan Qin, Wenjie Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)

Comments 12 pages, 9 figures, EMNLP 2024 Findings

2410.06967 2024-10-10 cs.CR cs.CY 78%

$\texttt{ModSCAN}$: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities

Yukun Jiang, Zheng Li, Xinyue Shen, Yugeng Liu, Michael Backes, Yang Zhang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments Accepted in EMNLP 2024. 29 pages, 22 figures

2406.11839 2024-10-08 cs.CV cs.AI cs.CL cs.LG 78%

mDPO: Conditional Preference Optimization for Multimodal Large Language Models

Fei Wang, Wenxuan Zhou, James Y. Huang, Nan Xu, Sheng Zhang, Hoifung Poon, Muhao Chen

专题命中幻觉与鲁棒性：multimodal large language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 Main Conference. Project website: https://feiwang96.github.io/mDPO

2407.02066 2024-10-07 cs.CL 78%

BiasDora: Exploring Hidden Biased Associations in Vision-Language Models

Chahat Raj, Anjishnu Mukherjee, Aylin Caliskan, Antonios Anastasopoulos, Ziwei Zhu

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments Accepted to EMNLP Findings 2024

2409.03966 2024-09-09 cs.RO 78%

Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts

Hongyi Chen, Yunchao Yao, Ruixuan Liu, Changliu Liu, Jeffrey Ichnowski

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

2403.01373 2024-05-07 cs.CL 78%

Quantity Matters: Towards Assessing and Mitigating Number Hallucination in Large Vision-Language Models

Huixuan Zhang, Junzhe Zhang, Xiaojun Wan

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

Comments 10 pages

2403.16645 2024-03-26 cs.HC 78%

Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations

Fan Li, Shanshan Feng, Yuqi Yan, Ching-Hung Lee, Yew Soon Ong

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)

Comments 10 pages,7 figures

2309.16426 2023-12-27 cs.RO 78%

QwenGrasp: A Usage of Large Vision-Language Model for Target-Oriented Grasping

Xinyu Chen, Jian Yang, Zonghan He, Haobin Yang, Qi Zhao, Yuhui Shi

专题命中幻觉与鲁棒性：vision-language model(title,abstract)

2506.03270 2026-08-04 cs.RO cs.AI 版本更新 77%

Grounded Vision-Language Interpreter for Long-Horizon Bimanual Task and Motion Planning

用于长 horizon 双臂任务与运动规划的接地视觉语言解释器

Jeremy Siburian, Keisuke Shirai, Cristian C. Beltran-Hernandez, Masashi Hamaya, Michael Görner, Atsushi Hashimoto

机构 * OMRON SINIC X Corporation（OMRON SINIC X公司）； The University of Tokyo（东京大学）； University of Hamburg（汉堡大学）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.AI

AI总结针对现有视觉语言机器人规划框架的黑箱缺陷与双臂任务探索不足问题，提出混合规划框架 ViLaIn-TAMP，经烹饪领域任务及实际双臂机器人系统验证，其性能优于基准方法。

Comments IROS 2026

2607.23704 2026-07-30 cs.RO cs.CV 版本更新 77%

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory

LabRobFail：化学自动驾驶实验室中机器人故障分析的基准

Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv, Ning Wang, Rui Li, Dongzhan Zhou, Weiyu Guo, Zhihui Wang, Wanli Ouyang

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV

AI总结针对化学自动驾驶实验室中机器人可靠性受化学实验特性限制、故障数据稀缺及评估协议缺乏等问题，引入LabRobFail框架，通过注入故障构建数据集，开发专门模型，提升故障检测等能力及下游任务成功率。

Comments Under review. Haobo Wang and Baoli Sun contributed equally. Code and data: https://github.com/Su-ISE-2001/SciRobo

2607.10985 2026-07-14 cs.CV 新提交 77%

MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration

MED-DSLC：通过域监督和对数校准进行多专家域分类

Zheng Zeng, Deepak Sridhar, Nuno Vasconcelos

机构 * University of California, San Diego（加利福尼亚大学圣地亚哥分校）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV

AI总结研究针对视觉语言模型在多域零样本分类中存在的问题，提出MED-DSLC方法，结合域监督训练和域内对数缩放，恢复全局对数可比性，是轻量级方案，经实验验证可提升准确率、鲁棒性和可扩展性。

Comments Accepted to ECCV 2026. Code is available at https://github.com/Leonard-Zeng/MED-DSLC

2502.07221 2026-07-02 cs.CV 版本更新 77%

Histopathology Multi-modal Embedding for Pathology Composed Retrieval

病理学组合检索的组织病理学多模态嵌入

Qifeng Zhou, Wenliang Zhong, Thao M. Dang, Hehuan Ma, Saiyang Na, Yuzhi Guo, Junzhou Huang

机构 * The University of Texas at Arlington（德克萨斯大学阿灵顿分校）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结提出HOMIE框架，将生成式多模态大语言模型适配为病理检索专家，解决组合查询中的架构、任务和领域不匹配问题，在病理组合检索基准上显著优于现有方法。

Comments Accepted by ECCV 2026

2606.23892 2026-06-24 cs.CV 新提交 77%

REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs

REALM: 面向物理世界视觉语言模型的统一红队基准

Yifei Zhao, Qian Lou, Mengxin Zheng

机构 * University of Central Florida（中佛罗里达大学）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV

AI总结提出REALM，首个统一红队基准，集成12种攻击方法、3种防御和13个视觉语言模型，通过代理目标生成管道实现公平比较，发现文本注入攻击最有效。

Comments 20 pages, 5 figures. Preprint

2606.21993 2026-06-23 cs.SE cs.CV 新提交 77%

From Driving Videos to Simulatable Scenarios

从驾驶视频到可模拟场景

Alexandre Levy, Ernest Valveny Llobet, Antonio Manuel López

机构 * Dept. Computer Science, Universitat Autònoma de Barcelona (UAB)（巴塞罗那自治大学计算机科学系）； Computer Vision Center (CVC), UAB（UAB计算机视觉中心）

专题命中幻觉与鲁棒性：VLM(abstract,abstract_cn)；vision language model(abstract)；分类 cs.CV

AI总结提出D-V2S框架，通过视觉语言模型和大语言模型从驾驶视频自动生成可模拟场景，实现90%语义元素保留和75%偏好率。

Comments 8 pages, 11 figures and Accepted for publication at the IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026

2606.17953 2026-06-17 cs.CV 新提交 77%

MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias

MLLMs 先正确后错误：追踪并纠正后层文本偏见

Xingming Li, Ao Cheng, Qiyao Sun, Xixiang He, Xuanyu Ji, Runke Huang, Qingyong Hu

机构 * National University of Defense Technology（国防科技大学）； Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Intelligent Game and Decision Lab（智能博弈与决策实验室）

专题命中幻觉与鲁棒性：grounding(abstract)；multimodal large language model(abstract)；MLLM(abstract_cn)；分类 cs.CV

AI总结发现多模态大语言模型在中间层形成正确视觉预测，但最终输出时被文本覆盖，通过检测预测方向变化（85%失败转向文本，89%成功转向视觉）提出无训练方法CALRD，在冲突基准上提升高达9.4%。

Comments Accepted at IJCAI 2026. 16 pages, 10 figures

2606.15608 2026-06-16 cs.CV 新提交 77%

On the Adversarial Robustness of Multimodal LLM Judges

多模态大语言模型评判器的对抗鲁棒性

Zihan Wang, Guansong Pang, Zelin Liu, Wenjun Miao, Jin Zheng, Xiao Bai

机构 * School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）； State Key Laboratory of Virtual Reality Technology and System, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）； State Key Laboratory of Software Development Environment, Jiangxi Research Institute, Beihang University（北京航空航天大学江西研究院软件开发环境国家重点实验室）； School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算机与信息系统学院）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结提出RobustMLLMJudge框架评估多模态大语言模型作为评判器时的对抗鲁棒性，并设计MGSIA攻击方法，通过语义诱导和高分流形对齐生成可迁移的分数膨胀扰动，揭示其脆弱性。

2606.12294 2026-06-11 cs.CV eess.IV 新提交 77%

Bridging the Modality Gap in Forensic Image Retrieval

弥合法医图像检索中的模态差距

Ricardo González-Gazapo, Annette Morales-González, Yoanna Martínez-Díaz, Heydi Méndez-Vázquez, Milton García-Borroto

机构 * Advanced Technologies Application Center (CENATAV)（先进技术应用中心（CENATAV））； Centro de Sistemas Complejos, Facultad de Física, Universidad de La Habana（哈瓦那大学物理学院复杂系统中心）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结提出统一检索框架，利用多模态大语言模型生成文本描述并结合视觉与文本特征融合，提升纹身、人脸素描等法医任务的检索精度与鲁棒性。

Comments 23 pages, 5 figures, paper submitted to Elsevier journal

2602.03890 2026-06-05 cs.CV 77%

4DPC$^2$hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping

4DPC$^2$hat: 面向动态点云理解的失败感知自举学习

Xindan Zhang, Weilong Yan, Yufei Shi, Xuerui Qiu, Tao He, Ying Li, Ming Li, Hehe Fan

机构 * University of Science and Technology of China（中国科学技术大学）

专题命中幻觉与鲁棒性：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结提出首个针对动态点云理解的多模态大语言模型4DPC$^2$hat，通过构建大规模跨模态数据集4DPC$^2$hat-200K和引入Mamba增强的时间推理模块及失败感知自举学习策略，显著提升了动作理解与时间推理能力。

Comments Accept by ICML 2026

1. 幻觉与鲁棒性 2260 篇

Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Models

Friction-Aware Safety Locomotion for Wheeled-legged Robots using Vision Language Models and Reinforcement Learning

RCG: Safety-Critical Scenario Generation for Robust Autonomous Driving via Real-World Crash Grounding

HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Model

Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack

Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense

Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow

Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective

A Novel MLLM-based Approach for Autonomous Driving in Different Weather Conditions

IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing

MACAROON: Training Vision-Language Models To Be Your Engaged Partners

Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models

$\texttt{ModSCAN}$: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities

mDPO: Conditional Preference Optimization for Multimodal Large Language Models

BiasDora: Exploring Hidden Biased Associations in Vision-Language Models

Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts

Quantity Matters: Towards Assessing and Mitigating Number Hallucination in Large Vision-Language Models

Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations

QwenGrasp: A Usage of Large Vision-Language Model for Target-Oriented Grasping

Grounded Vision-Language Interpreter for Long-Horizon Bimanual Task and Motion Planning

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory

MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration

Histopathology Multi-modal Embedding for Pathology Composed Retrieval

REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs

From Driving Videos to Simulatable Scenarios

MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias

On the Adversarial Robustness of Multimodal LLM Judges

Bridging the Modality Gap in Forensic Image Retrieval

4DPC$^2$hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping