视觉大模型 / VLM - arXivDaily 专题

2506.21874 2025-06-30 cs.CR cs.AI 70%

On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling

Stanley Wu, Ronik Bhaskar, Anna Yoo Jeong Ha, Shawn Shan, Haitao Zheng, Ben Y. Zhao

机构 * University of Chicago（芝加哥大学）

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments ACM Conference on Computer and Communications Security 2025

2503.01584 2025-06-30 cs.AI 70%

SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

Cansu Sancaktar, Christian Gumbsch, Andrii Zadaianchuk, Pavel Kolev, Georg Martius

机构 * Autonomous Learning, University of Tübingen（图宾根大学自主学习研究所）； Empirical Inference, Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）； Neuro-Cognitive Modeling, University of Tübingen（图宾根大学神经认知建模研究所）； VISLab, University of Amsterdam（阿姆斯特丹大学视觉实验室）

专题命中幻觉与鲁棒性：vision language model(abstract)；VLM(abstract)；分类 cs.AI

Comments ICML 2025 camera-ready version. Project webpage at https://sites.google.com/view/sensei-paper

2506.00618 2025-06-23 cs.AI 70%

RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents

Jingyi Yang, Shuai Shao, Dongrui Liu, Jing Shao

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）； University of Science and Technology of China（中国科学技术大学）； Shanghai Jiao Tong University（上海交通大学）

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments 40 pages, 6 figures, Project Page: https://yjyddq.github.io/RiOSWorld.github.io/

2506.13038 2025-06-18 cs.CV cs.MM 70%

HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMs

Zijian Zhang, Xuecheng Wu, Danlei Huang, Siyu Yan, Chong Peng, Xuezhi Cao

机构 * Xi'an Jiaotong University（西安交通大学）； East China Normal University（华东师范大学）

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2506.07564 2025-06-12 cs.AI cs.CL 70%

SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems

Peiran Li, Xinkai Zou, Zhuohang Wu, Ruifeng Li, Shuo Xing, Hanwen Zheng, Zhikai Hu, Yuping Wang, Haoxi Li, Qin Yuan, Yingmo Zhang, Zhengzhong Tu

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments Former versions either contain unrelated content or cannot be properly converted to PDF

2506.07971 2025-06-10 cs.CV 70%

CyberV: Cybernetics for Test-time Scaling in Video Understanding

Jiahao Meng, Shuyang Sun, Yue Tan, Lu Qi, Yunhai Tong, Xiangtai Li, Longyin Wen

机构 * Peking University（北京大学）； ByteDance（字节跳动）

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2505.18657 2025-05-27 cs.AI 70%

MLLMs are Deeply Affected by Modality Bias

Xu Zheng, Chenfei Liao, Yuqian Fu, Kaiyu Lei, Yuanhuiyi Lyu, Lutao Jiang, Bin Ren, Jialei Chen, Jiawen Wang, Chengxin Li, Linfeng Zhang, Danda Pani Paudel, Xuanjing Huang, Yu-Gang Jiang, Nicu Sebe, Dacheng Tao, Luc Van Gool, Xuming Hu

机构 * HKUST(GZ)（香港科技大学（广州））； CSE, HKUST（香港科技大学计算机科学与工程系）； Xi’an Jiaotong University（西安交通大学）； University of Pisa, IT（比萨大学）； University of Trento, IT（特伦特大学）； Nagoya University（名古屋大学）； China University of Mining & Technology, Beijing（中国矿业大学（北京））； Tongji University（同济大学）； SPIC Energy Science and Technology Research Institute（SPIC能源科学与技术研究院）； Shanghai Jiao Tong University（上海交通大学）； Fudan University（复旦大学）； College of Computing & Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

2504.20468 2025-05-08 cs.CV 70%

Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception

Yuanchen Wu, Lu Zhang, Hang Yao, Junlong Du, Ke Yan, Shouhong Ding, Yunsheng Wu, Xiaoqiang Li

机构 * School of Computer Engineering & Science, Shanghai University（上海大学计算机工程与科学学院）； Tencent YouTu Lab（腾讯优图实验室）

专题命中幻觉与鲁棒性：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

2505.03153 2025-05-07 cs.CV 70%

Robust Fairness Vision-Language Learning for Medical Image Analysis

Sparsh Bansal, Mingyang Wu, Xin Wang, Shu Hu

机构 * Purdue University（普渡大学）； University at Albany, State University of New York（纽约州立大学阿尔巴尼分校）

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2504.21040 2025-05-01 cs.CV 70%

Can a Large Language Model Assess Urban Design Quality? Evaluating Walkability Metrics Across Expertise Levels

Chenyi Cai, Kosuke Kuriyama, Youlong Gu, Filip Biljecki, Pieter Herthogs

机构 * Singapore-ETH Centre, Future Cities Lab Global Programme, CREATE campus（新加坡-ETH中心，未来城市实验室全球计划，CREATE校区）； National University of Singapore（新加坡国立大学）； Takenaka Corporation（Takenaka公司）； Department of Architecture, National University of Singapore（新加坡国立大学建筑系）

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2410.06172 2025-04-24 cs.AI cs.CL 70%

Multimodal Situational Safety

Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, Xin Eric Wang

机构 * University of California, Santa Cruz（加州大学圣克ruz分校）； University of California, Berkeley（加州大学伯克利分校）

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments ICLR 2025 Camera Ready

2504.13690 2025-04-22 cs.CV 70%

Analysing the Robustness of Vision-Language-Models to Common Corruptions

Muhammad Usama, Syeda Aishah Asim, Syed Bilal Ali, Syed Talal Wasim, Umair Bin Mansoor

机构 * Department of Electrical Engineering, DHA Suffa University Karachi（电气工程系，达哈萨夫大学卡拉奇）； Department of Computer Science, University of Bonn（计算机科学系，波恩大学）

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2304.10592, arXiv:2301.12597 by other authors

2503.23715 2025-04-01 cs.CV 70%

HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generation

Kun Liu, Qi Liu, Xinchen Liu, Jie Li, Yongdong Zhang, Jiebo Luo, Xiaodong He, Wu Liu

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments CVPR 2025

2502.14882 2025-03-26 cs.CV 70%

CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs

Insu Han, Zeliang Zhang, Zhiyuan Wang, Yifan Zhu, Susan Liang, Jiani Liu, Haiting Lin, Mingjie Zhao, Chenliang Xu, Kun Wan, Wentian Zhao

专题命中幻觉与鲁棒性：InternVL(abstract)；multimodal large language model(abstract)；分类 cs.CV

2503.14189 2025-03-19 cs.CL cs.CV 70%

Towards Harmless Multimodal Assistants with Blind Preference Optimization

Yongqi Li, Lu Yang, Jian Wang, Runyang You, Wenjie Li, Liqiang Nie

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2503.10613 2025-03-14 cs.CV 70%

CoSTA$\ast$: Cost-Sensitive Toolpath Agent for Multi-turn Image Editing

Advait Gupta, NandaKiran Velaga, Dang Nguyen, Tianyi Zhou

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2503.07216 2025-03-12 cs.LG 70%

FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates

Sangwoo Park, Seanie Lee, Byungjoo Kim, Sung Ju Hwang

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.LG

Comments Preprint

2501.19164 2025-02-24 cs.CV 70%

Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs

Kejia Zhang, Keda Tao, Jiasheng Tang, Huan Wang

专题命中幻觉与鲁棒性：vision-language model(abstract)；grounding(abstract)；分类 cs.CV

2502.14486 2025-02-21 cs.CR cs.AI cs.CL 70%

How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation

Zhuohang Long, Siyuan Wang, Shujun Liu, Yuhang Lai, Xuanjing Huang, Zhongyu Wei

专题命中幻觉与鲁棒性：vision-language model(abstract)；LLaVA(abstract)；分类 cs.AI

2502.09818 2025-02-17 cs.CV 70%

On the robustness of multimodal language model towards distractions

Ming Liu, Hao Chen, Jindong Wang, Wensheng Zhang

专题命中幻觉与鲁棒性：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

2502.06390 2025-02-12 cs.CV 70%

When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

Aobotao Dai, Xinyu Ma, Lei Chen, Songze Li, Lin Wang

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2502.02438 2025-02-05 cs.CR cs.AI 70%

Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

Yaling Shen, Zhixiong Zhuang, Kun Yuan, Maria-Irina Nicolae, Nassir Navab, Nicolas Padoy, Mario Fritz

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Accepted at AAAI 2025

2501.04947 2025-01-10 cs.CV 70%

Seeing with Partial Certainty: Conformal Prediction for Robotic Scene Recognition in Built Environments

Yifan Xu, Vineet Kamat, Carol Menassa

专题命中幻觉与鲁棒性：vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 10 pages, 4 Figures

2412.12222 2024-12-18 cs.CV 70%

Endangered Alert: A Field-Validated Self-Training Scheme for Detecting and Protecting Threatened Wildlife on Roads and Roadsides

Kunming Li, Mao Shan, Stephany Berrio Perez, Katie Luo, Stewart Worrall

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 8 pages, 8 figures

2409.14674 2024-09-24 cs.RO cs.CL cs.CV 70%

RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning

Yinpei Dai, Jayjun Lee, Nima Fazeli, Joyce Chai

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Project Website: https://rich-language-failure-recovery.github.io

2405.12255 2024-05-24 eess.IV cs.CV 70%

Mammo-CLIP: A Vision Language Foundation Model to Enhance Data Efficiency and Robustness in Mammography

Shantanu Ghosh, Clare B. Poynton, Shyam Visweswaran, Kayhan Batmanghelich

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments MICCAI 2024, early accept, top 11%

2404.11864 2024-04-25 cs.CV 70%

Progressive Multi-modal Conditional Prompt Tuning

Xiaoyu Qiu, Hao Feng, Yuechen Wang, Wengang Zhou, Houqiang Li

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2308.10450 2024-03-12 cs.CV 70%

COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain Adaptation

Xinghong Liu, Yi Zhou, Tao Zhou, Chun-Mei Feng, Ling Shao

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2210.05335 2023-07-21 cs.CV cs.CL cs.MM 70%

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model

Yatai Ji, Junjie Wang, Yuan Gong, Lin Zhang, Yanru Zhu, Hongfa Wang, Jiaxing Zhang, Tetsuya Sakai, Yujiu Yang

专题命中幻觉与鲁棒性：visual reasoning(abstract)；visual question answering(abstract)；分类 cs.CV

Comments CVPR 2023 Main Track Long Paper

2507.19362 2025-12-02 cs.CV cs.AI cs.CL cs.CY cs.LG 69%

LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferences

LOTUS: 一种用于从质量到社会偏见和用户偏好的详细图像描述的排行榜

Yusuke Hirota, Boyi Li, Ryo Hachiuma, Yueh-Hua Wu, Boris Ivanovic, Yuta Nakashima, Marco Pavone, Yejin Choi, Yu-Chiang Frank Wang, Chao-Han Huck Yang

机构 * NVIDIA Research（NVIDIA研究）

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG；VLM(comments)

AI总结 LOTUS是一种用于评估详细图像描述质量、偏见和社会偏见的排行榜，通过定制标准满足不同用户偏好，揭示了模型在不同评估标准上的表现差异。

Comments Accepted to ACL 2025. Leaderboard: huggingface.co/spaces/nvidia/lotus-vlm-bias-leaderboard

1. 幻觉与鲁棒性 2260 篇

On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling

SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents

HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMs

SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems

CyberV: Cybernetics for Test-time Scaling in Video Understanding

MLLMs are Deeply Affected by Modality Bias

Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception

Robust Fairness Vision-Language Learning for Medical Image Analysis

Can a Large Language Model Assess Urban Design Quality? Evaluating Walkability Metrics Across Expertise Levels

Multimodal Situational Safety

Analysing the Robustness of Vision-Language-Models to Common Corruptions

HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generation

CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs

Towards Harmless Multimodal Assistants with Blind Preference Optimization

CoSTA$\ast$: Cost-Sensitive Toolpath Agent for Multi-turn Image Editing

FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates

Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs

How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation

On the robustness of multimodal language model towards distractions

When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

Seeing with Partial Certainty: Conformal Prediction for Robotic Scene Recognition in Built Environments

Endangered Alert: A Field-Validated Self-Training Scheme for Detecting and Protecting Threatened Wildlife on Roads and Roadsides

RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning

Mammo-CLIP: A Vision Language Foundation Model to Enhance Data Efficiency and Robustness in Mammography

Progressive Multi-modal Conditional Prompt Tuning

COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain Adaptation

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model

LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferences