视觉大模型 / VLM

Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs

Chongjun Tu, Peng Ye, Dongzhan Zhou, Lei Bai, Gang Yu, Tao Chen, Wanli Ouyang

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Urban Safety Perception Through the Lens of Large Multimodal Models: A Persona-based Approach

Ciro Beneduce, Bruno Lepri, Massimiliano Luca

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.AI

From Screens to Scenes: A Survey of Embodied AI in Healthcare

Yihao Liu, Xu Cao, Tingting Chen, Yankai Jiang, Junjie You, Minghua Wu, Xiaosong Wang, Mengling Feng, Yaochu Jin, Jintai Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments 56 pages, 11 figures, manuscript accepted by Information Fusion

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, Lihua Zhang

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment

Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan Ö. Arık, Tomas Pfister

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Published in ICLR 2025

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

Wenpeng Xing, Minghao Li, Mohan Li, Meng Han

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Dysca: A Dynamic and Scalable Benchmark for Evaluating Perception Ability of LVLMs

Jie Zhang, Zhongqi Wang, Mengqi Lei, Zheng Yuan, Bei Yan, Shiguang Shan, Xilin Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ICLR2025

Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation

Hieu Nguyen, Zihao He, Shoumik Atul Gandre, Ujjwal Pasupulety, Sharanya Kumari Shivakumar, Kristina Lerman

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences

Shanshan Han, Salman Avestimehr, Chaoyang He

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments arXiv admin note: text overlap with arXiv:2406.10847

MARAGE: Transferable Multi-Model Adversarial Attack for Retrieval-Augmented Generation Data Extraction

Xiao Hu, Eric Liu, Weizhou Wang, Xiangyu Guo, David Lie

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports

Romain Hardy, Sung Eun Kim, Du Hyun Ro, Pranav Rajpurkar

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Comments Accepted to AIMedHealth 10 pages, 5 figures

ViG-Bias: Visually Grounded Bias Discovery and Mitigation

Badr-Eddine Marani, Mohamed Hanini, Nihitha Malayarukil, Stergios Christodoulidis, Maria Vakalopoulou, Enzo Ferrante

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV

Comments ECCV 2024

Breaking Barriers in Physical-World Adversarial Examples: Improving Robustness and Transferability via Robust Feature

Yichen Wang, Yuxuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, Minghui Li

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by AAAI2025

Craft: Cross-modal Aligned Features Improve Robustness of Prompt Tuning

Jingchen Sun, Rohan Sharma, Vishnu Suresh Lokhande, Changyou Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to WACV 2025

CRoF: CLIP-based Robust Few-shot Learning on Noisy Labels

Shizhuo Deng, Bowen Han, Jiaqi Chen, Hao Wang, Dongyue Chen, Tong Jia

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?

Shuo Chen, Zhen Han, Bailan He, Zifeng Ding, Wenqian Yu, Philip Torr, Volker Tresp, Jindong Gu

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.LG

Comments technical report; update code repo link

GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration

Kaiyi Huang, Yukun Huang, Xuefei Ning, Zinan Lin, Yu Wang, Xihui Liu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Comments Project website: https://karine-h.github.io/GenMAC/

WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions

Seyedali Mohammadi, Edward Raff, Jinendra Malekar, Vedant Palit, Francis Ferraro, Manas Gaur

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments Accepted in BlackboxNLP @ EMNLP 2024

Journal ref Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 364-388, November 2024, Miami, Florida, US. Association for Computational Linguistics

Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration

Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, Tao Jiang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

Comments Repo: NeurIPS2024_SPV-MIA" target="_blank" rel="noopener">https://github.com/tsinghua-fib-lab/NeurIPS2024_SPV-MIA

Journal ref The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)

Leveraging Pre-trained Models for FF-to-FFPE Histopathological Image Translation

Qilai Zhang, Jiawen Li, Peiran Liao, Jiali Hu, Tian Guan, Anjia Han, Yonghong He

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at IEEE BIBM 2024

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

Hao Zhang, Jin-Jian Xu, Hong-Wei Cui, Lin Li, Yaowen Yang, Chao-Sheng Tang, Niklas Boers

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Comments accpeted by IEEE Geoscience and Remote Sensing Magazine

Journal ref IEEE Geoscience and Remote Sensing Magazine, 2024

Enhancing Visual Classification using Comparative Descriptors

Hankyeol Lee, Gawon Seo, Wonseok Choi, Geunyoung Jung, Kyungwoo Song, Jiyoung Jung

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by WACV 2025

Improving Bilingual Capabilities of Language Models to Support Diverse Linguistic Practices in Education

Anand Syamkumar, Nora Tseng, Kaycie Barron, Shanglin Yang, Shamya Karumbaiah, Rheeya Uppal, Junjie Hu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.AI

TorchOpera: A Compound AI System for LLM Safety

Shanshan Han, Zijian Hu, Alay Dilipbhai Shah, Han Jin, Yuhang Yao, Dimitris Stripelis, Zhaozhuo Xu, Chaoyang He

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting

Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, Hanwang Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2024 Spotlight

ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users

Guanlin Li, Kangjie Chen, Shudong Zhang, Jie Zhang, Tianwei Zhang

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.AI

Comments Accepted by NeurIPS 2024

HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding

Fan Yuan, Chi Qin, Xiaogang Xu, Piji Li

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at Main Conference of EMNLP 2024

Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP

Zeliang Zhang, Zhuo Liu, Mingqian Feng, Chenliang Xu

专题命中 幻觉与鲁棒性 ：visual question answering(abstract)；分类 cs.CV

Comments Short paper. Accepted by the Findings of EMNLP 2024

CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Camera

Jingpei Lu, Zekai Liang, Tristin Xie, Florian Ritcher, Shan Lin, Sainan Liu, Michael C. Yip

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 7 pages, 5 figures, project website: https://sites.google.com/ucsd.edu/ctrnet-x

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding

Xiaoyu Liang, Jiayuan Yu, Lianrui Mu, Jiedong Zhuang, Jiaqi Hu, Yuchen Yang, Jiangnan Ye, Lu Lu, Jian Chen, Haoji Hu

专题命中 幻觉与鲁棒性 ：visual question answering(abstract)；分类 cs.CV

Comments PRCV