视觉大模型 / VLM - arXivDaily 专题

2504.10000 2025-04-15 cs.CR cs.AI cs.CL cs.CV cs.LG 67%

Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?

Yanbo Wang, Jiyang Guan, Jian Liang, Ran He

专题命中幻觉与鲁棒性：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2025, codes in process

2503.15166 2025-04-15 cs.CV cs.AI cs.CL cs.LG cs.MM 67%

Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU

Àlex Pujol Vidal, Sergio Escalera, Kamal Nasrollahi, Thomas B. Moeslund

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Preprint

2503.23388 2025-04-01 cs.CV cs.AI cs.LG cs.MM 67%

COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptation

Fanding Huang, Jingyan Jiang, Qinting Jiang, Hebei Li, Faisal Nadeem Khan, Zhi Wang

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2025

2412.02479 2025-03-28 cs.CV cs.AI cs.CR cs.LG 67%

OODFace: Benchmarking Robustness of Face Recognition under Common Corruptions and Appearance Variations

Caixin Kang, Yubo Chen, Shouwei Ruan, Shiji Zhao, Ruochen Zhang, Jiayi Wang, Shan Fu, Xingxing Wei

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2503.11360 2025-03-17 cs.CV cs.AI cs.LG 67%

PARIC: Probabilistic Attention Regularization for Language Guided Image Classification from Pre-trained Vison Language Models

Mayank Nautiyal, Stela Arranz Gheorghe, Kristiana Stefa, Li Ju, Ida-Maria Sintorn, Prashant Singh

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2503.08162 2025-03-12 cs.RO cs.CL 67%

FASIONAD++ : Integrating High-Level Instruction and Information Bottleneck in FAt-Slow fusION Systems for Enhanced Safety in Autonomous Driving with Adaptive Feedback

Kangan Qian, Ziang Luo, Sicong Jiang, Zilin Huang, Jinyu Miao, Zhikun Ma, Tianze Zhu, Jiayin Li, Yangfan He, Zheng Fu, Yining Shi, Boyue Wang, Hezhe Lin, Ziyu Chen, Jiangbo Yu, Xinyu Jiao, Mengmeng Yang, Kun Jiang, Diange Yang

专题命中幻觉与鲁棒性：vision language model(abstract)；VLM(abstract)

Comments 8 pages, 4 figures

2503.06626 2025-03-11 cs.CV cs.AI cs.LG 67%

DiffCLIP: Differential Attention Meets CLIP

Hasan Abed Al Kader Hammoud, Bernard Ghanem

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Under review

2503.02865 2025-03-06 cs.CL 67%

FairSense-AI: Responsible AI Meets Sustainability

Shaina Raza, Mukund Sayeeganesh Chettiar, Matin Yousefabadi, Tahniat Khan, Marcelo Lotif

专题命中幻觉与鲁棒性：vision-language model(abstract)；vision language model(abstract)

2410.01417 2025-03-04 cs.CV cs.AI cs.CL cs.LG 67%

The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs

Hong Li, Nanxi Li, Yuanjie Chen, Jianbin Zhu, Qinlu Guo, Cewu Lu, Yong-Lu Li

专题命中幻觉与鲁棒性：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICLR 2025. Project page: https://mvig-rhos.com/llm_inception

2502.17710 2025-02-26 cs.AI cs.CL cs.CV cs.LG 67%

Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures

Akhila Yerukola, Saadia Gabriel, Nanyun Peng, Maarten Sap

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 40 pages, 49 figures

2502.11969 2025-02-18 cs.AI cs.CV cs.LG 67%

Learning Generalizable Prompt for CLIP with Class Similarity Knowledge

Sehun Jung, Hyang-won Lee

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2405.13573 2025-02-13 cs.RO 67%

Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance

Kaifeng Zhang, Zhao-Heng Yin, Weirui Ye, Yang Gao

专题命中幻觉与鲁棒性：vision-language model(abstract)；VLM(abstract)

2412.07979 2024-12-12 cs.LG cs.AI cs.CV 67%

AmCLR: Unified Augmented Learning for Cross-Modal Representations

Ajay Jagannath, Aayush Upadhyay, Anant Mehta

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 16 pages, 2 figures

2412.03178 2024-12-05 cs.AI cs.CV cs.LG 67%

Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation

Gianni Franchi, Dat Nguyen Trong, Nacim Belkhir, Guoxuan Xia, Andrea Pilzer

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 28 pages and 22 figures

2405.00693 2024-11-28 cs.RO cs.CL 67%

Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls

Jesse Atuhurra

专题命中幻觉与鲁棒性：vision language model(abstract)；VLM(abstract)

2407.02352 2024-10-30 cs.CL 67%

Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification

Pritish Sahu, Karan Sikka, Ajay Divakaran

专题命中幻觉与鲁棒性：visual language model(abstract)；grounding(abstract)

2410.11302 2024-10-16 cs.CV cs.AI cs.CL cs.LG 67%

Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs

Shuo Li, Tao Ji, Xiaoran Fan, Linsheng Lu, Leyi Yang, Yuming Yang, Zhiheng Xi, Rui Zheng, Yuran Wang, Xiaohui Zhao, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中幻觉与鲁棒性：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2407.11229 2024-10-07 cs.CL cs.AI cs.CV cs.HC cs.LG 67%

Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness

Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, Dan Roth

专题命中幻觉与鲁棒性：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 22 pages, 9 Tables, 5 figures, 22 examples

2408.01959 2024-08-29 cs.CV cs.AI cs.CL cs.CY cs.LG 67%

Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI

Robert Wolfe, Aayushi Dangol, Alexis Hiniker, Bill Howe

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at Artificial Intelligence, Ethics, and Society 2024

2312.17174 2024-06-25 cs.CV cs.AI cs.LG 67%

Visual Explanations of Image-Text Representations via Multi-Modal Information Bottleneck Attribution

Ying Wang, Tim G. J. Rudner, Andrew Gordon Wilson

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published in Advances in Neural Information Processing Systems 36 (NeurIPS 2023)

2405.19538 2024-06-05 cs.CL cs.AI cs.CV cs.LG 67%

CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats

Pierre Chambon, Jean-Benoit Delbrouck, Thomas Sounack, Shih-Cheng Huang, Zhihong Chen, Maya Varma, Steven QH Truong, Chu The Chuong, Curtis P. Langlotz

专题命中幻觉与鲁棒性：vision language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 13 pages Updated title

2405.19098 2024-05-30 cs.LG cs.AI cs.CR cs.CV stat.ML 67%

Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior

Shuyu Cheng, Yibo Miao, Yinpeng Dong, Xiao Yang, Xiao-Shan Gao, Jun Zhu

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICML 2024

2104.00170 2024-04-24 cs.LG cs.AI cs.CV stat.ML 67%

Are Bias Mitigation Techniques for Deep Learning Effective?

Robik Shrestha, Kushal Kafle, Christopher Kanan

专题命中幻觉与鲁棒性：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published in WACV 2022 under the title "An Investigation of Critical Issues in Bias Mitigation Techniques"

2308.01313 2024-03-19 cs.CV cs.AI cs.CL cs.LG 67%

PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts

Bang An, Sicheng Zhu, Michael-Andrei Panaitescu-Liess, Chaithanya Kumar Mummadi, Furong Huang

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICLR 2024

2301.13096 2024-03-12 cs.CV cs.AI cs.LG 67%

Language-Driven Anchors for Zero-Shot Adversarial Robustness

Xiao Li, Wei Zhang, Yining Liu, Zhanhao Hu, Bo Zhang, Xiaolin Hu

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by CVPR 2024

2305.14998 2024-02-07 cs.CL cs.AI cs.CV cs.LG 67%

An Examination of the Robustness of Reference-Free Image Captioning Evaluation Metrics

Saba Ahmadi, Aishwarya Agrawal

专题命中幻觉与鲁棒性：grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

2309.11751 2023-10-17 cs.CV cs.AI cs.CR cs.LG 67%

How Robust is Google's Bard to Adversarial Image Attacks?

Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Technical report

2110.11205 2023-01-25 cs.LG cs.AI cs.CL cs.CV 67%

Robustness through Data Augmentation Loss Consistency

Tianjian Huang, Shaunak Halbe, Chinnadhurai Sankar, Pooyan Amini, Satwik Kottur, Alborz Geramifard, Meisam Razaviyayn, Ahmad Beirami

专题命中幻觉与鲁棒性：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 40 pages

1902.03380 2021-10-11 cs.CV cs.AI cs.LG cs.SC 67%

When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networks

Chao-Han Huck Yang, Yi-Chieh Liu, Pin-Yu Chen, Xiaoli Ma, Yi-Chang James Tsai

专题命中幻觉与鲁棒性：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Noted our camera-ready version has changed the title. "When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networks" as the v3 official paper title in IEEE Proceeding. Please use it in your formal reference. Accepted at IEEE ICIP 2019. Pytorch code has released on https://github.com/jjaacckkyy63/Causal-Intervention-AE-wAdvImg

Journal ref 2019 26th IEEE International Conference on Image Processing (ICIP). IEEE

2608.10513 2026-08-12 cs.CV cs.AI 新提交 66%

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

SafeCap：通过图像字幕强化学习提升大型视觉语言模型（LVLM）的安全性

Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang Zheng

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI；VLM(comments)

AI总结 SafeCap是一种通过图像字幕强化学习提升LVLM安全性的框架，在多模态安全基准上安全得分提升3.7-19.0个百分点，且视觉效用相当或更优，性能优于安全SFT等方法。

Comments 16pages, 4 figures. Preprint. Project page: https://safe-vlm.github.io/SafeCap/ ; code: https://github.com/Safe-VLM/SafeCap

1. 幻觉与鲁棒性 2272 篇

Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?

Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU

COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptation

OODFace: Benchmarking Robustness of Face Recognition under Common Corruptions and Appearance Variations

PARIC: Probabilistic Attention Regularization for Language Guided Image Classification from Pre-trained Vison Language Models

FASIONAD++ : Integrating High-Level Instruction and Information Bottleneck in FAt-Slow fusION Systems for Enhanced Safety in Autonomous Driving with Adaptive Feedback

DiffCLIP: Differential Attention Meets CLIP

FairSense-AI: Responsible AI Meets Sustainability

The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs

Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures

Learning Generalizable Prompt for CLIP with Class Similarity Knowledge

Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance

AmCLR: Unified Augmented Learning for Cross-Modal Representations

Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation

Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls

Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification

Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs

Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness

Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI

Visual Explanations of Image-Text Representations via Multi-Modal Information Bottleneck Attribution

CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats

Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior

Are Bias Mitigation Techniques for Deep Learning Effective?

PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts

Language-Driven Anchors for Zero-Shot Adversarial Robustness

An Examination of the Robustness of Reference-Free Image Captioning Evaluation Metrics

How Robust is Google's Bard to Adversarial Image Attacks?

Robustness through Data Augmentation Loss Consistency

When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networks

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning