多模态大模型 - arXivDaily 专题

2510.15162 2025-10-20 cs.CV cs.CL 86%

Train a Unified Multimodal Data Quality Classifier with Synthetic Data

Weizhi Wang, Rongmei Lin, Shiyang Li, Colin Lockard, Ritesh Sarkhel, Sanket Lokegaonkar, Jingbo Shang, Xifeng Yan, Nasser Zalmout, Xian Li

机构 * UC Santa Barbara（加州大学圣芭芭拉分校）； Amazon Stores Foundational AI（亚马逊商店基础人工智能）； UC San Diego（加州大学圣地亚哥分校）

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2025 Findings

2508.06434 2025-09-26 cs.CV cs.AI 86%

CLIPin: A Non-contrastive Plug-in to CLIP for Multimodal Semantic Alignment

Shengzhu Yang, Jiawei Du, Shuai Lu, Weihang Zhang, Ningli Wang, Huiqi Li

机构 * Beijing Institute of Technology（北京理工大学）； Beijing Tongren Hospital（北京同仁医院）

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2507.20994 2025-07-29 cs.CV cs.AI 86%

Security Tensors as a Cross-Modal Bridge: Extending Text-Aligned Safety to Vision in LVLM

Shen Li, Liuyi Yao, Wujia Niu, Lan Zhang, Yaliang Li

机构 * University of Science and Technology of China（中国科学技术大学）

专题命中图文多模态：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Codes and data are available at https://github.com/listen0425/Security-Tensors

2507.08590 2025-07-14 cs.MM cs.CV 86%

Visual Semantic Description Generation with MLLMs for Image-Text Matching

Junyu Chen, Yihua Gao, Mingyong Li

机构 * Chongqing Normal University（重庆师范大学）

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICME2025 oral

2506.23639 2025-07-01 cs.CV cs.AI 86%

Unified Multimodal Understanding via Byte-Pair Visual Encoding

Wanpeng Zhang, Yicheng Feng, Hao Luo, Yijiang Li, Zihao Yue, Sipeng Zheng, Zongqing Lu

机构 * Peking University（北京大学）； UC San Diego（加州大学圣地亚哥分校）； Renmin University of China（中国人民大学）； BeingBeyond

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

2501.15370 2025-05-30 cs.CV cs.AI 86%

Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis

Robinson Umeike, Neil Getty, Fangfang Xia, Rick Stevens

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 4 Pages, 4 Figures, 1 Table

2406.08707 2025-05-30 cs.CL cs.CV 86%

mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus

Matthieu Futeral, Armel Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot

机构 * Inria（法国国家信息与自动化技术研究院）； Département d’informatique de l’ENS, CNRS, PSL Research University（巴黎高等师范学院计算机科学系、国家科学研究中心、巴黎综合理工研究学院）； Institut du développement et des ressources en informatique scientifique, CNRS（科学信息发展与资源研究所、国家科学研究中心）； Sorbonne Université（索邦大学）； Common Crawl Foundation（Common Crawl基金会）； Université Paris-Saclay（巴黎萨克雷大学）

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments ACL 2025 (Findings)

2306.11207 2025-01-14 cs.CV cs.CL cs.LG 86%

Quilt-1M: One Million Image-Text Pairs for Histopathology

Wisdom Oluchi Ikezogwo, Mehmet Saygin Seyfioglu, Fatemeh Ghezloo, Dylan Stefan Chan Geva, Fatwir Sheikh Mohammed, Pavan Kumar Anand, Ranjay Krishna, Linda Shapiro

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

2407.08303 2024-11-26 cs.CV cs.AI 86%

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception

Xiaotong Li, Fan Zhang, Haiwen Diao, Yueze Wang, Xinlong Wang, Ling-Yu Duan

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024. Project is available at https://github.com/baaivision/DenseFusion

2411.00304 2024-11-04 cs.CV cs.MM 86%

Unified Generative and Discriminative Training for Multi-modal Large Language Models

Wei Chow, Juncheng Li, Qifan Yu, Kaihang Pan, Hao Fei, Zhiqi Ge, Shuai Yang, Siliang Tang, Hanwang Zhang, Qianru Sun

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；MLLM(abstract)；image-text(abstract)

2410.11963 2024-10-17 cs.CV cs.AI 86%

CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning

Qingqing Cao, Mahyar Najibi, Sachin Mehta

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

2409.20424 2024-10-01 cs.CV cs.AI 86%

World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering

Jiacong Wang, Bohong Wu, Haiyong Jiang, Xun Zhou, Xin Xiao, Haoyuan Guo, Jun Xiao

专题命中图文多模态：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at EMNLP 2024 Main Conference, 16pages

2409.12568 2024-09-20 cs.CV cs.MM 86%

InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning

Xiaotian Han, Yiren Jian, Xuefeng Hu, Haogeng Liu, Yiqi Wang, Qihang Fan, Yuang Ai, Huaibo Huang, Ran He, Zhenheng Yang, Quanzeng You

专题命中图文多模态：multimodal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

2402.02544 2024-07-17 cs.CV cs.AI cs.LG 86%

LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model

Dilxat Muhtar, Zhenshi Li, Feng Gu, Xueliang Zhang, Pengfeng Xiao

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 36 pages, 10 figures. Github https://github.com/NJU-LHRS/LHRS-Bot

2406.07588 2024-07-02 cs.MM cs.CL 86%

AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning

Jun Gao, Qian Qiao, Ziqiang Cao, Zili Wang, Wenjie Li

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CL、cs.MM

2406.12479 2024-06-19 cs.CV cs.AI 86%

RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding

Linrui Xu, Ling Zhao, Wang Guo, Qiujun Li, Kewang Long, Kaiqi Zou, Yuhan Wang, Haifeng Li

专题命中图文多模态：multimodal(title,abstract)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 6 figures, 4 tables

2406.03496 2024-06-06 cs.CL cs.AI cs.LG 86%

Wings: Learning Multimodal LLMs without Text-only Forgetting

Yi-Kai Zhang, Shiyin Lu, Yang Li, Yanqing Ma, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, De-Chuan Zhan, Han-Jia Ye

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CL、cs.AI

2404.18114 2024-04-30 cs.CV cs.MM 86%

Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching

Haiwen Diao, Ying Zhang, Shang Gao, Xiang Ruan, Huchuan Lu

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 12 pages, 9 figures, Accepted by TIP2024

2401.15896 2024-02-06 cs.CV cs.AI 86%

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Qingpei Guo, Furong Xu, Hanxiao Zhang, Wang Ren, Ziping Ma, Lin Ju, Jian Wang, Jingdong Chen, Ming Yang

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

2310.13619 2023-10-23 cs.CL cs.CV 86%

Semi-supervised multimodal coreference resolution in image narrations

Arushi Goel, Basura Fernando, Frank Keller, Hakan Bilen

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Long paper at EMNLP'23-Main

2306.14824 2023-07-14 cs.CL cs.CV 86%

Kosmos-2: Grounding Multimodal Large Language Models to the World

Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments 20 pages

2302.06605 2023-05-23 cs.CV cs.CL 86%

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

Haoyu Lu, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Wei Zhan, Masayoshi Tomizuka, Mingyu Ding

专题命中图文多模态：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

2211.09371 2023-03-21 cs.CV cs.MM 86%

CapEnrich: Enriching Caption Semantics for Web Images via Cross-modal Pre-trained Knowledge

Linli Yao, Weijing Chen, Qin Jin

专题命中图文多模态：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Comments Accepted to WWW2023

2203.11480 2022-05-03 cs.CV cs.CL 86%

WuDaoMM: A large-scale Multi-Modal Dataset for Pre-training models

Sha Yuan, Shuai Zhao, Jiahong Leng, Zhao Xue, Hanyu Zhao, Peiyu Liu, Zheng Gong, Wayne Xin Zhao, Junyi Li, Jie Tang

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Some data problems cannot be obtained

2506.21710 2025-10-30 cs.CV 86%

FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering

Liangyu Zhong, Fabio Rosenthal, Joachim Sicking, Fabian Hüger, Thorsten Bagdonat, Hanno Gottschalk, Leo Schwinn

机构 * Technical University of Berlin（柏林技术大学）； Technical University of Munich（慕尼黑技术大学）； CARIAD SE ； Volkswagen AG（大众集团）

专题命中图文多模态：MLLM(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2025 - main track. Project page: https://focus-mllm-vqa.github.io/

2103.06561 2021-07-09 cs.CV cs.IR 86%

WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training

Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao, Ruichen Li, Yida Zhao, Liang Zhang, Yuqing Song, Xin Hong, Wanqing Cui, Danyang Hou, Yingyan Li, Junyi Li, Peiyu Liu, Zheng Gong, Chuhao Jin, Yuchong Sun, Shizhe Chen, Zhiwu Lu, Zhicheng Dou, Qin Jin, Yanyan Lan, Wayne Xin Zhao, Ruihua Song, Ji-Rong Wen

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments This paper is the outcome of the Chinese multi-modal pre-training project called 'WenLan'

2502.18200 2025-05-30 eess.SP 86%

Zero-Shot Semantic Communication with Multimodal Foundation Models

Jiangjing Hu, Haotian Wu, Wenjing Zhang, Fengyu Wang, Wenjun Xu, Hui Gao, Deniz Gündüz

专题命中图文多模态：multimodal(title)；multimodal foundation model(title)；multi-modal(abstract)

2403.08783 2024-03-15 cs.CV cs.AI cs.CL 86%

Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformation

Fatma Shalabi, Huy H. Nguyen, Hichem Felouat, Ching-Chun Chang, Isao Echizen

专题命中图文多模态：multimodal(title)；image-text(title)；分类 cs.CV、cs.CL、cs.AI

Comments 8 pages, 2 figures, conference

2608.24325 2026-08-26 cs.AI 新提交 85%

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

SonarLLM：一种用于水下感知的原生声呐-光学多模态大语言模型

Cong Su, longxuan ma, Ling Dong, Guofeng Tang, Weijie Yin, Haohui Chen, Zhengtao Yu

机构 * Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）； Yunnan Key Laboratory of Artificial Intelligence（云南省人工智能重点实验室）

专题命中图文多模态：multimodal(title)；MLLM(abstract,abstract_cn)；cross-modal(abstract)；分类 cs.AI

AI总结针对现有多模态大语言模型不适用于水下声呐-光学感知的问题，提出SonarLLM模型，结合专用编码器与分层融合机制，在SonarBench基准上实现了优异的水下感知性能，凸显了声呐的互补价值。

2608.21443 2026-08-25 cs.CV 新提交 85%

Text-Guided Visual Dependency Graph Learning with Cross-Modal Attention Priors

基于跨模态注意力先验的文本引导视觉依赖图学习

Fei Wang, Yutong Zhang, Yang Ye, Jinxian Chen, Wang Wenshuai, Xiong Wang

专题命中图文多模态：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

AI总结提出CM-GLasso框架，结合文本可视化、交叉注意力蒸馏和联合ADMM，在八个基准上取得最优分类与分割性能，可生成可解释的稀疏条件依赖图。

1. 图文多模态 4688 篇

Train a Unified Multimodal Data Quality Classifier with Synthetic Data

CLIPin: A Non-contrastive Plug-in to CLIP for Multimodal Semantic Alignment

Security Tensors as a Cross-Modal Bridge: Extending Text-Aligned Safety to Vision in LVLM

Visual Semantic Description Generation with MLLMs for Image-Text Matching

Unified Multimodal Understanding via Byte-Pair Visual Encoding

Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis

mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus

Quilt-1M: One Million Image-Text Pairs for Histopathology

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception

Unified Generative and Discriminative Training for Multi-modal Large Language Models

CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning

World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering

InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning

LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model

AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning

RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding

Wings: Learning Multimodal LLMs without Text-only Forgetting

Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Semi-supervised multimodal coreference resolution in image narrations

Kosmos-2: Grounding Multimodal Large Language Models to the World

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

CapEnrich: Enriching Caption Semantics for Web Images via Cross-modal Pre-trained Knowledge

WuDaoMM: A large-scale Multi-Modal Dataset for Pre-training models

FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering

WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training

Zero-Shot Semantic Communication with Multimodal Foundation Models

Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformation

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

Text-Guided Visual Dependency Graph Learning with Cross-Modal Attention Priors