视觉大模型 / VLM

Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving

Yuhang Lu, Yichen Yao, Jiadong Tu, Jiangnan Shao, Yuexin Ma, Xinge Zhu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination

Dingchen Yang, Bowen Cao, Guang Chen, Changjun Jiang

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Image-to-Text Logic Jailbreak: Your Imagination can Help You Do Anything

Xiaotian Zou, Ke Li, Yongkang Chen

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Physics-Inspired Generative Models in Medical Imaging: A Review

Dennis Hein, Afshin Bozorgpour, Dorit Merhof, Ge Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Conditional Prototype Rectification Prompt Learning

Haoxing Chen, Yaohui Li, Zizheng Huang, Yan Hong, Zhuoer Xu, Zhangxuan Gu, Jun Lan, Huijia Zhu, Weiqiang Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors

Lin Guan, Yifan Zhou, Denis Liu, Yantian Zha, Heni Ben Amor, Subbarao Kambhampati

专题命中 幻觉与鲁棒性 ：VLM(abstract)；分类 cs.AI

Comments Published as a conference paper at COLM 2024

Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs

Shi Liu, Kecheng Zheng, Wei Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Data Processing Techniques for Modern Multimodal Models

Yinheng Li, Han Ding, Hang Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding

Quan Kong, Yuki Kawana, Rajat Saini, Ashutosh Kumar, Jingjing Pan, Ta Gu, Yohei Ozao, Balazs Opra, David C. Anastasiu, Yoichi Sato, Norimasa Kobori

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV24. Website: https://woven-visionai.github.io/wts-dataset-homepage/

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Yanting Yang, Minghao Chen, Qibo Qiu, Jiahao Wu, Wenxiao Wang, Binbin Lin, Ziyu Guan, Xiaofei He

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV 2024 camera-ready

LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition

Teng Wang, Lingquan Meng, Lei Cheng, Changyin Sun

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Learning Robust 3D Representation from CLIP via Dual Denoising

Shuqing Luo, Bowen Qu, Wei Gao

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV

Evidential Concept Embedding Models: Towards Reliable Concept Explanations for Skin Disease Diagnosis

Yibo Gao, Zheyao Gao, Xin Gao, Yuanye Liu, Bomin Wang, Xiahai Zhuang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments accepted by MICCAI 2024

Coarse-to-Fine Concept Bottleneck Models

Konstantinos P. Panousis, Dino Ienco, Diego Marcos

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

Guardrails for avoiding harmful medical product recommendations and off-label promotion in generative AI models

Daniel Lopez-Martinez

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments CVPR 2024 Responsible Generative AI (ReGenAI) workshop

Versatile Backdoor Attack with Visible, Semantic, Sample-Specific, and Compatible Triggers

Ruotong Wang, Hongrui Chen, Zihao Zhu, Li Liu, Baoyuan Wu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 23 pages, 21 figures, 18 tables

VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models

Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, Zilong Zheng

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking

Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Towards Event-oriented Long Video Understanding

Yifan Du, Kun Zhou, Yuqi Huo, Yifan Li, Wayne Xin Zhao, Haoyu Lu, Zijia Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Work on progress

From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment

Yusuke Hirota, Ryo Hachiuma, Chao-Han Huck Yang, Yuta Nakashima

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Hallucination Mitigation Prompts Long-term Video Understanding

Yiwei Sun, Zhihang Liu, Chuanbin Liu, Bowei Pu, Zhihan Zhang, Hongtao Xie

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Spuriousness-Aware Meta-Learning for Learning Robust Classifiers

Guangtao Zheng, Wenqian Ye, Aidong Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to KDD 2024

M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark

Wei Song, Yadong Li, Jianhua Xu, Guowei Wu, Lingfeng Ming, Kexin Yi, Weihua Luo, Houyi Li, Yi Du, Fangda Guo, Kaicheng Yu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.AI

Situated Ground Truths: Enhancing Bias-Aware AI by Situating Data Labels with SituAnnotate

Delfina Sol Martinez Pandiani, Valentina Presutti

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments Author preprint

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, Tong Zhang

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation

Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, Di Lin, Kaicheng Yu

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Comments Project Page: https://westlake-autolab.github.io/delphi.github.io/, 8 figures

ED-SAM: An Efficient Diffusion Sampling Approach to Domain Generalization in Vision-Language Foundation Models

Thanh-Dat Truong, Xin Li, Bhiksha Raj, Jackson Cothren, Khoa Luu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Automated Multi-level Preference for MLLMs

Mengxi Zhang, Wenhao Wu, Yu Lu, Yuxin Song, Kang Rong, Huanjin Yao, Jianbo Zhao, Fanglong Liu, Yifan Sun, Haocheng Feng, Jingdong Wang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Preprint

Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography

Nhat Chung, Sensen Gao, Tuan-Anh Vu, Jie Zhang, Aishan Liu, Yun Lin, Jin Song Dong, Qing Guo

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Comments 12 pages, 5 tables, 5 figures, work in progress

BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity

Andrew F. Luo, Margaret M. Henderson, Michael J. Tarr, Leila Wehbe

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

Comments ICLR 2024. Project page: https://www.cs.cmu.edu/~afluo/BrainSCUBA