多模态大模型 - arXivDaily 专题

2512.06811 2025-12-09 cs.CV cs.AI cs.LG cs.MM 82%

RMAdapter: Reconstruction-based Multi-Modal Adapter for Vision-Language Models

RMAdapter: 基于重建的多模态适配器用于视觉-语言模型

Xiang Lin, Weixin Li, Shu Guo, Lihong Wang, Di Huang

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

AI总结 RMAdapter通过双分支架构平衡通用与任务特定知识，提升视觉-语言模型在多模态迁移学习中的性能。

Comments Accepted by AAAI 2026(Oral)

2512.03463 2025-12-04 cs.CV cs.AI cs.CL 82%

Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models

文本打印图像：为以文本为中心的大型视觉-语言模型训练弥合图像-文本模态差距

Shojiro Yamabe, Futa Waseda, Daiki Shiono, Tsubasa Takahashi

机构 * Turing Inc.（图灵公司）； Institute of Science Tokyo（东京科学研究院）； The University of Tokyo（东京大学）； Tohoku University（东北大学）

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结本研究提出文本打印图像（TPI）技术，通过生成合成图像弥合图像-文本模态差距，提升以文本为中心的大型视觉-语言模型训练效果。

2504.14359 2025-11-12 cs.CV cs.AI cs.CL 82%

A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modeling

Kyle Buettner, Jacob T. Emmerson, Adriana Kovashka

机构 * Intelligent Systems Program（智能系统项目）； Department of Computer Science（计算机科学系）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at IJCNLP-AACL 2025 (Main)

2504.07110 2025-11-07 cs.IR cs.LG 82%

DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDash

Omkar Gurjar, Kin Sum Liu, Praveen Kolli, Utsaw Kumar, Mandar Rahurkar

机构 * DoorDash, Inc.（DoorDash公司）

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；image-text(abstract)

2506.12116 2025-10-28 cs.CL cs.AI cs.CV 82%

Unsupervised Document and Template Clustering using Multimodal Embeddings

Phillipe R. Sampaio, Helene Maxcici

机构 * BNP Paribas Cardif Nanterre（BNP巴黎银行卡迪夫纳特尔尔分校）； BNP Paribas Paris（BNP巴黎银行巴黎）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 24 pages, 12 figures

2505.02152 2025-10-09 cs.RO 82%

Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions

Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, Mingyu Ding

机构 * Shanghai Jiao Tong University（上海交通大学）； UC Berkeley（伯克利大学）； UNC, Chapel Hill（北卡罗来纳大学教堂山分校）

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)

2502.15969 2025-08-26 cs.CV cs.AI cs.CL 82%

Forgotten Polygons: Multimodal Large Language Models are Shape-Blind

William Rudman, Michal Golovanevsky, Amir Bar, Vedant Palit, Yann LeCun, Carsten Eickhoff, Ritambhara Singh

机构 * Brown University（布朗大学）； Tel Aviv University（特拉维夫大学）； IIT Kharagpur（印度理工学院卡里帕尔分校）； New York University（纽约大学）； University of Tübingen（图宾根大学）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2508.00456 2025-08-18 eess.SP 82%

When Vision-Language Model (VLM) Meets Beam Prediction: A Multimodal Contrastive Learning Framework

Ji Wang, Bin Tang, Jian Xiao, Qimei Cui, Xingwang Li, Tony Q. S. Quek

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)

2505.05422 2025-08-18 cs.CV cs.AI cs.CL 82%

TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation

Haokun Lin, Teng Wang, Yixiao Ge, Yuying Ge, Zhichao Lu, Ying Wei, Qingfu Zhang, Zhenan Sun, Ying Shan

机构 * ARC Lab, Tencent PCG（腾讯PCG广告实验室）； City University of Hong Kong（香港城市大学）； Zhejiang University（浙江大学）； NLPR & MAIS, Institute of Automation, CAS, Beijing（自动化研究所）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical Report

2507.04377 2025-08-12 cs.CV cs.CL cs.MM 82%

Multi-Modal Semantic Parsing for the Interpretation of Tombstone Inscriptions

Xiao Zhang, Johan Bos

机构 * University of Groningen（格罗宁根大学）

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments ACMMM 2025

2508.05684 2025-08-11 cs.CR cs.LG 82%

MM-FusionNet: Context-Aware Dynamic Fusion for Multi-modal Fake News Detection with Large Vision-Language Models

Junhao He, Tianyu Liu, Jingyuan Zhao, Benjamin Turner

机构 * Huaiyin Institute of Technology（淮阴职业技术学院）； Universidad Autónoma de Asunción（阿斯unción自治大学）

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)

2410.08642 2025-08-05 cs.SI cs.CL cs.CV cs.MM 82%

More than Memes: A Multimodal Topic Modeling Approach to Conspiracy Theories on Telegram

Elisabeth Steffen

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 12 pages, 10 figures

Journal ref Proceedings of the Nineteenth International AAAI Conference on Web and Social Media, Vol. 19 (2025)

2507.07151 2025-07-11 cs.CV cs.AI cs.CL 82%

Robust Multimodal Large Language Models Against Modality Conflict

Zongmeng Zhang, Wengang Zhou, Jie Zhao, Houqiang Li

机构 * School of Artificial Intelligence（人工智能学院）； Data Science, University of Science（数据科学，科学大学）； Department of Electronic Engineering（电子工程系）； Information Science, University of Science（信息科学，科学大学）； Huawei Technologies Co., Ltd.（华为技术有限公司）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ICML 2025

2503.04065 2025-06-27 cs.CV cs.AI cs.CL 82%

PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks

Feng Ni, Kui Huang, Yao Lu, Wenyu Lv, Guanzhong Wang, Zeyu Chen, Yi Liu

机构 * PaddlePaddle Team, Baidu Inc.（百度公司）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2504.17353 2025-06-13 cs.CL cs.CV cs.MM 82%

M-MRE: Extending the Mutual Reinforcement Effect to Multimodal Information Extraction

Chengguang Gan, Zhixi Cai, Yanbin Wei, Yunhao Liang, Shiwen Ni, Tatsunori Mori

机构 * Yokohama National University（Yokohama国立大学）； Monash University（墨尔本大学）； Southern University of Science and Technology（南方科技大学）； Hong Kong University of Science and Technology（香港科学大学）； University of Chinese Academy of Sciences（中国科学院大学）； Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

2506.07936 2025-06-10 cs.CV cs.AI cs.CL cs.LG 82%

Mimicking or Reasoning: Rethinking Multi-Modal In-Context Learning in Vision-Language Models

Chengyue Huang, Yuchen Zhu, Sichen Zhu, Jingyun Xiao, Moises Andrade, Shivang Chopra, Zsolt Kira

机构 * Georgia Institute of Technology（佐治亚理工学院）

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2412.00142 2025-06-10 cs.CV cs.AI cs.CL 82%

Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features

Chancharik Mitra, Brandon Huang, Tianning Chai, Zhiqiu Lin, Assaf Arbelle, Rogerio Feris, Leonid Karlinsky, Trevor Darrell, Deva Ramanan, Roei Herzig

机构 * Carnegie Mellon University（卡内基梅隆大学）； University of California, Berkeley（加州大学伯克利分校）； IBM Research（IBM研究院）； MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2412.06141 2025-06-05 cs.CV cs.AI cs.CL cs.LG 82%

MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization

Kangyu Zhu, Peng Xia, Yun Li, Hongtu Zhu, Sheng Wang, Huaxiu Yao

机构 * UNC Chapel-Hill（UNC夏洛特-希尔分校）； Brown University（布朗大学）； University of Washington（华盛顿大学）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ICML 2025

2503.14604 2025-06-02 cs.CV cs.AI cs.CL 82%

Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives

Sara Sarto, Marcella Cornia, Rita Cucchiara

机构 * University of Modena and Reggio Emilia（摩德纳和雷吉奥艾米利亚大学）； IIT-CNR（意大利国家研究委员会IIT）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments IJCAI 2025. Repo GitHub: https://github.com/aimagelab/awesome-captioning-evaluation

2505.22943 2025-05-30 cs.CL cs.AI cs.CV cs.LG cs.SD 82%

Can LLMs Deceive CLIP? Benchmarking Adversarial Compositionality of Pre-trained Multimodal Representation via Text Updates

Jaewoo Ahn, Heeseung Yun, Dayoon Ko, Gunhee Kim

机构 * Seoul National University（首尔国立大学）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL 2025 Main. Code is released at https://vision.snu.ac.kr/projects/mac

2505.16446 2025-05-23 cs.LG 82%

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Zhaoxin Wang, Handing Wang, Cong Tian, Yaochu Jin

机构 * Xidian University（西安电子科技大学）； Westlake University（西湖大学）

专题命中图文多模态：cross-modal(title,abstract)；multimodal(abstract)

2505.05040 2025-05-09 cs.CL cs.AI cs.CV 82%

Image-Text Relation Prediction for Multilingual Tweets

Matīss Rikters, Edison Marrese-Taylor

机构 * Artificial Intelligence Research Center（人工智能研究中心）； National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Published in Proceedings of the 1st Workshop on Nordic-Baltic Responsible Evaluation and Alignment of Language, NoDaLiDa - Baltic HLT 2025

2505.04911 2025-05-09 cs.CV cs.AI cs.CL 82%

SpatialPrompting: Keyframe-driven Zero-Shot Spatial Reasoning with Off-the-Shelf Multimodal Large Language Models

Shun Taguchi, Hideki Deguchi, Takumi Hamazaki, Hiroyuki Sakai

机构 * Toyota Central R&D Labs., Inc.（丰田中央研究实验室）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 18 pages, 11 figures

2505.00150 2025-05-02 cs.CV cs.AI cs.CL 82%

Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models

Minh-Hao Van, Xintao Wu

机构 * Department of EECS University of Arkanasas Fayetteville（电子工程与科学系阿肯色大学法洛维尔分校）

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2412.00876 2025-03-24 cs.CV cs.AI cs.CL cs.LG 82%

Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification

Wenxuan Huang, Zijie Zhai, Yunhang Shen, Shaosheng Cao, Fei Zhao, Xiangfeng Xu, Zheyu Ye, Yao Hu, Shaohui Lin

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ICLR 2025. Code is available at https://github.com/Osilly/dynamic_llava

2403.09974 2025-03-24 cs.CV cs.AI cs.CL cs.LG 82%

GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery

Enguang Wang, Zhimao Peng, Zhengyuan Xie, Fei Yang, Xialei Liu, Ming-Ming Cheng

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2025

2503.10582 2025-03-18 cs.CV cs.AI cs.CL 82%

VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search

Yiming Jia, Jiachen Li, Xiang Yue, Bo Li, Ping Nie, Kai Zou, Wenhu Chen

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical Report

2503.10857 2025-03-17 cs.GR cs.AI cs.CL cs.CV 82%

Towards Understanding Graphical Perception in Large Multimodal Models

Kai Zhang, Jianwei Yang, Jeevana Priya Inala, Chandan Singh, Jianfeng Gao, Yu Su, Chenglong Wang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Work in Progress

2502.00094 2025-02-06 cs.CV cs.AI cs.CL cs.HC cs.LG 82%

AIN: The Arabic INclusive Large Multimodal Model

Ahmed Heakl, Sara Ghaboura, Omkar Thawkar, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 20 pages, 16 figures, ACL

2410.10879 2024-12-11 cs.LG cs.AI cs.CL cs.CV 82%

Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word Frequency

Mingliang Liang, Martha Larson

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV、cs.CL、cs.AI

1. 图文多模态 4682 篇

RMAdapter: Reconstruction-based Multi-Modal Adapter for Vision-Language Models

Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models

A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modeling

DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDash

Unsupervised Document and Template Clustering using Multimodal Embeddings

Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions

Forgotten Polygons: Multimodal Large Language Models are Shape-Blind

When Vision-Language Model (VLM) Meets Beam Prediction: A Multimodal Contrastive Learning Framework

TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation

Multi-Modal Semantic Parsing for the Interpretation of Tombstone Inscriptions

MM-FusionNet: Context-Aware Dynamic Fusion for Multi-modal Fake News Detection with Large Vision-Language Models

More than Memes: A Multimodal Topic Modeling Approach to Conspiracy Theories on Telegram

Robust Multimodal Large Language Models Against Modality Conflict

PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks

M-MRE: Extending the Mutual Reinforcement Effect to Multimodal Information Extraction

Mimicking or Reasoning: Rethinking Multi-Modal In-Context Learning in Vision-Language Models

Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features

MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization

Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives

Can LLMs Deceive CLIP? Benchmarking Adversarial Compositionality of Pre-trained Multimodal Representation via Text Updates

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Image-Text Relation Prediction for Multilingual Tweets

SpatialPrompting: Keyframe-driven Zero-Shot Spatial Reasoning with Off-the-Shelf Multimodal Large Language Models

Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models

Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification

GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery

VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search

Towards Understanding Graphical Perception in Large Multimodal Models

AIN: The Arabic INclusive Large Multimodal Model

Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word Frequency