视觉大模型 / VLM

Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness

Sibo Wang, Jie Zhang, Zheng Yuan, Shiguang Shan

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models

Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, Nicholas Carlini

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

HallE-Control: Controlling Object Hallucination in Large Multimodal Models

Bohan Zhai, Shijia Yang, Chenfeng Xu, Sheng Shen, Kurt Keutzer, Chunyuan Li, Manling Li

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Comments Our code is publicly available at https://github.com/bronyayang/HallE_Control

MemeCraft: Contextual and Stance-Driven Multimodal Meme Generation

Han Wang, Roy Ka-Wei Lee

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.AI

Comments 8 pages, 7 figures, ACM MM 2024

OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, Nenghai Yu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Comments CVPR 2024, code is available at https://github.com/shikiw/OPERA

$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations

Xiang Li, Kai Qiu, Jinglu Wang, Xiaohao Xu, Rita Singh, Kashu Yamazak, Hao Chen, Xiaonan Huang, Bhiksha Raj

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.CV

Comments Code and dataset will be released at https://github.com/lxa9867/r2bench

Rethinking Model Ensemble in Transfer-based Adversarial Attacks

Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding

Haogeng Liu, Quanzeng You, Xiaotian Han, Yiqi Wang, Bohan Zhai, Yongfei Liu, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization

Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, Conghui He

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Project Website: https://opendatalab.github.io/HA-DPO, Code: https://github.com/opendatalab/HA-DPO

TeG-DG: Textually Guided Domain Generalization for Face Anti-Spoofing

Lianrui Mu, Jianhong Bai, Xiaoxuan He, Jiangnan Ye, Xiaoyu Liang, Yuchen Yang, Jiedong Zhuang, Haoji Hu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift

Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li

专题命中 幻觉与鲁棒性 ：visual reasoning(abstract)；分类 cs.CV

Comments Accepted by Journal of Data-centric Machine Learning Research (DMLR) 2024

VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution

Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shtedritski, Hannah Rose Kirk

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS Datasets and Benchmarks 2023. Data and code available at https://github.com/oxai/visogender

AutoRepo: A general framework for multi-modal LLM-based automated construction reporting

Hongxu Pu, Xincong Yang, Jing Li, Runhao Guo, Heng Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments We believe that keeping this version of the paper publicly available may lead to confusion or misinterpretation regarding our current research direction and findings

GELDA: A generative language annotation framework to reveal visual biases in datasets

Krish Kabra, Kathleen M. Lewis, Guha Balakrishnan

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 21 pages, 15 figures, 9 tables

Hallucination Augmented Recitations for Language Models

Abdullatif Köksal, Renat Aksitov, Chung-Ching Chang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Read-only Prompt Optimization for Vision-Language Few-shot Learning

Dongjun Lee, Seokwon Song, Jihee Suh, Joonmyung Choi, Sanghyeok Lee, Hyunwoo J. Kim

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at ICCV2023

Uniformly Distributed Category Prototype-Guided Vision-Language Framework for Long-Tail Recognition

Siming Fu, Xiaoxuan He, Xinpeng Ding, Yuchen Cao, Hualiang Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 11pages, 5figures

Journal ref ACM MM2023

ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification

Mohammad Reza Taesiri, Giang Nguyen, Sarra Habchi, Cor-Paul Bezemer, Anh Nguyen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2023 Track on Datasets and Benchmarks

Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association

Qiyu Wu, Mengjie Zhao, Yutong He, Lang Huang, Junya Ono, Hiromi Wakaki, Yuki Mitsufuji

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.CV

Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping

Adam Rashid, Satvik Sharma, Chung Min Kim, Justin Kerr, Lawrence Chen, Angjoo Kanazawa, Ken Goldberg

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments See the project website at: lerftogo.github.io

Implicit Neural Representation for Cooperative Low-light Image Enhancement

Shuzhou Yang, Moxuan Ding, Yanmin Wu, Zihan Li, Jian Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models

Yimu Wang, Peng Shi, Hongyang Zhang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments 19 pages

Visual Adversarial Examples Jailbreak Aligned Large Language Models

Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.LG

Transferable Decoding with Visual Entities for Zero-Shot Image Captioning

Junjie Fei, Teng Wang, Jinrui Zhang, Zhenyu He, Chengjie Wang, Feng Zheng

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment

Yongrae Jo, Seongyun Lee, Aiden SJ Lee, Hyunji Lee, Hanseok Oh, Minjoon Seo

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks

Nils Lukas, Florian Kerschbaum

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

Comments Preprint

Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptions

Ian Huang, Vrishab Krishna, Omoruyi Atekha, Leonidas Guibas

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks

Haiyang Xu, Qinghao Ye, Xuan Wu, Ming Yan, Yuan Miao, Jiabo Ye, Guohai Xu, Anwen Hu, Yaya Shi, Guangwei Xu, Chenliang Li, Qi Qian, Maofei Que, Ji Zhang, Xiao Zeng, Fei Huang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Working in progress

Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets

Brandon Smith, Miguel Farinha, Siobhan Mackenzie Hall, Hannah Rose Kirk, Aleksandar Shtedritski, Max Bain

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Github: https://github.com/oxai/debias-gensynth

Understanding Zero-Shot Adversarial Robustness for Large-Scale Models

Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, Carl Vondrick

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV