多模态大模型

Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data?

Che Liu, Zhongwei Wan, Haozhe Wang, Yinda Chen, Talha Qaiser, Chen Jin, Fariba Yousefi, Nikolay Burlutskiy, Rossella Arcucci

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Under Review

SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features

Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, Xiaohua Zhai

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Model checkpoints are available at https://github.com/google-research/big_vision/tree/main/big_vision/configs/proj/image_text/README_siglip2.md

PLPHP: Per-Layer Per-Head Vision Token Pruning for Efficient Large Vision-Language Models

Yu Meng, Kaiyuan Li, Chenran Huang, Chen Gao, Xinlei Chen, Yong Li, Xiaoping Zhang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 8 figures

Understanding and Rectifying Safety Perception Distortion in VLMs

Xiaohan Zou, Jian Kang, George Kesidis, Lu Lin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Fake It Till You Make It: Using Synthetic Data and Domain Knowledge for Improved Text-Based Learning for LGE Detection

Athira J Jacob, Puneet Sharma, Daniel Rueckert

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Poster at Workshop on Large Language Models and Generative AI for Health at AAAI 2025

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Zikang Liu, Kun Zhou, Wayne Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments under review

MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation

Taha Koleilat, Hojat Asgariandehkordi, Hassan Rivaz, Yiming Xiao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 2 figures, 6 tables

Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations

Kangyu Zhu, Ziyuan Qin, Huahui Yi, Zekun Jiang, Qicheng Lao, Shaoting Zhang, Kang Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NAACL 2025 Main Conference

BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks

Yunhan Zhao, Xiang Zheng, Lin Luo, Yige Li, Xingjun Ma, Yu-Gang Jiang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Journal ref ICLR 2025, BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks. In Proceedings of the International Conference on Learning Representations (ICLR), 2025

Evaluating Vision-Language Models for Emotion Recognition

Sree Bhattacharyya, James Z. Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NAACL 2025 Findings

Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generation

Jihyo Kim, Seulbi Lee, Sangheum Hwang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICLR 2025. The first two authors contributed equally

MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems

Zifeng Zhu, Mengzhao Jia, Zhihan Zhang, Lang Li, Meng Jiang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments NAACL 2025, 19 pages, 10 figures

Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

Marco Mistretta, Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, Andrew D. Bagdanov

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at ICLR 2025

What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning

Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song, Ji-Rong Wen

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by COLING2025

AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis

Basit Alawode, Iyyakutti Iyappan Ganapathi, Sajid Javed, Naoufel Werghi, Mohammed Bennamoun, Arif Mahmood

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

CLIP-DQA: Blindly Evaluating Dehazed Images from Global and Local Perspectives Using CLIP

Yirui Zeng, Jun Fu, Hadi Amirpour, Huasheng Wang, Guanghui Yue, Hantao Liu, Ying Chen, Wei Zhou

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ISCAS 2025 (Oral)

Vision-Language Modeling in PET/CT for Visual Grounding of Positive Findings

Zachary Huemann, Samuel Church, Joshua D. Warner, Daniel Tran, Xin Tie, Alan B McMillan, Junjie Hu, Steve Y. Cho, Meghan Lubner, Tyler J. Bradshaw

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification

Xiangyu Sun, Xiaoguang Zou, Yuanquan Wu, Guotai Wang, Shaoting Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments This paper has been accepted for presentation at the 2025 IEEE International Symposium on Biomedical Imaging (ISBI 2025)

Large Models in Dialogue for Active Perception and Anomaly Detection

Tzoulio Chamiti, Nikolaos Passalis, Anastasios Tefas

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to International Conference of Pattern Recognition (ICPR 2024)

Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities

Shounak Datta, Dhanasekar Sundararaman

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Amala Sanjay Deshmukh, Tuomas Rintamaki, Matthieu Le, Ilia Karmanov, Lukas Voegtle, Philipp Fischer, De-An Huang, Timo Roman, Tong Lu, Jose M. Alvarez, Bryan Catanzaro, Jan Kautz, Andrew Tao, Guilin Liu, Zhiding Yu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Global Semantic-Guided Sub-image Feature Weight Allocation in High-Resolution Large Vision-Language Models

Yuxuan Liang, Xu Li, Xiaolei Chen, Haotian Chen, Yi Zheng, Chenghang Lai, Bin Li, Xiangyang Xue

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 10 figures and tables

Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models

Jakob Krogh Petersen, Valdemar Licht, Mads Nielsen, Asbjørn Munk

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures. To be published in ISBI 2025

Pix2Cap-COCO: Advancing Visual Comprehension via Pixel-Level Captioning

Zuyao You, Junke Wang, Lingyu Kong, Bo He, Zuxuan Wu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VisMin: Visual Minimal-Change Understanding

Rabiul Awal, Saba Ahmadi, Le Zhang, Aishwarya Agrawal

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted at NeurIPS 2024. Project URL at https://vismin.net/

Patch is Enough: Naturalistic Adversarial Patch against Vision-Language Pre-training Models

Dehong Kong, Siyuan Liang, Xiaopeng Zhu, Yuansheng Zhong, Wenqi Ren

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments accepted by Visual Intelligence

Journal ref Visual Intelligence, 2024, Vol 2, article no.17

MedFILIP: Medical Fine-grained Language-Image Pre-training

Xinjie Liang, Xiangyu Li, Fanding Li, Jie Jiang, Qing Dong, Wei Wang, Kuanquan Wang, Suyu Dong, Gongning Luo, Shuo Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 5 figures, IEEE Journal of Biomedical and Health Informatics 2025

Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model

Huan Ma, Yan Zhu, Changqing Zhang, Peilin Zhao, Baoyuan Wu, Long-Kai Huang, Qinghua Hu, Bingzhe Wu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Visual question answering: from early developments to recent advances -- a survey

Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Sunil Aryal, Imran Razzak, Hakim Hacid

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 20 papers