多模态大模型 - arXivDaily 专题

2505.23091 2025-06-24 cs.AI cs.CL 84%

Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models

Zeyu Liu, Yuhang Liu, Guanghao Zhu, Congkai Xie, Zhen Li, Jianbo Yuan, Xinyao Wang, Qing Li, Shing-Chi Cheung, Shengyu Zhang, Fei Wu, Hongxia Yang

机构 * The Hong Kong Polytechnic University（香港理工大学）； Zhejiang University（浙江大学）； University of Electronic Science and Technology of China（电子科技大学）； Reallm Labs（Reallm 实验室）； Amazon（亚马逊）； The Hong Kong University of Science and Technology（香港理工大学）

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

2412.19663 2025-06-24 cs.CV cs.AI cs.GR 84%

CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs

Siyu Wang, Cailian Chen, Xinyi Le, Qimin Xu, Lei Xu, Yanzhou Zhang, Jie Yang

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2025 (Vol. 39, No. 8), pages 7880-7888. DOI: 10.1609/aaai.v39i8.32849

Journal ref Proc. of the AAAI Conf. on Artificial Intelligence, 39(8):7880-7888, 2025

2505.14260 2025-05-21 cs.CV cs.AI 84%

Speculative Decoding Reimagined for Multimodal Large Language Models

Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室、中华人民共和国教育部、厦门大学）

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 12 pages

2505.14071 2025-05-21 cs.LG cs.CL cs.CV 84%

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models

Woody Haosheng Gan, Deqing Fu, Julian Asilis, Ollie Liu, Dani Yogatama, Vatsal Sharan, Robin Jia, Willie Neiswanger

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

2504.12773 2025-04-18 cs.CL cs.AI 84%

Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration

Yicheng Pan, Zhenrong Zhang, Pengfei Hu, Jiefeng Ma, Jun Du, Jianshu Zhang, Quan Liu, Jianqing Gao, Feng Ma

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 5 figures

2407.19651 2025-02-18 cs.CV cs.LG cs.MM 84%

Bridging Compressed Image Latents and Multimodal Large Language Models

Chia-Hao Kao, Cheng Chien, Yu-Jen Tseng, Yi-Hsin Chen, Alessandro Gnutti, Shao-Yuan Lo, Wen-Hsiao Peng, Riccardo Leonardi

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICLR 2025

2411.12591 2024-11-20 cs.CV cs.AI 84%

Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, Lichao Sun

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2410.06765 2024-10-10 cs.CL cs.CV 84%

To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models

Junyan Lin, Haoran Chen, Dawei Zhu, Xiaoyu Shen

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Accepted to EMNLP 2024 Main Conference

2409.04999 2024-09-10 cs.CV cs.MM 84%

Visual Grounding with Multi-modal Conditional Adaptation

Ruilin Yao, Shengwu Xiong, Yichen Zhao, Yi Rong

专题命中其他多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM MM 2024 [Oral]

2408.02888 2024-08-07 cs.CV cs.AI 84%

VizECGNet: Visual ECG Image Network for Cardiovascular Diseases Classification with Multi-Modal Training and Knowledge Distillation

Ju-Hyeon Nam, Seo-Hyung Park, Su Jung Kim, Sang-Chul Lee

专题命中其他多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted in International Conference on Image Processing (ICIP) 2024

2407.08583 2024-08-06 cs.AI cs.CV cs.LG 84%

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中其他多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Ongoing work. 21 pages. Related materials are continually maintained and available at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md

2407.16198 2024-07-24 cs.CV cs.AI 84%

INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model

Yiwei Ma, Zhibin Wang, Xiaoshuai Sun, Weihuang Lin, Qiang Zhou, Jiayi Ji, Rongrong Ji

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2403.12488 2024-07-24 cs.CV cs.AI 84%

DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM

Yixuan Wu, Yizhou Wang, Shixiang Tang, Wenhao Wu, Tong He, Wanli Ouyang, Philip Torr, Jian Wu

专题命中其他多模态：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

2407.01518 2024-07-02 cs.CV cs.AI cs.LG 84%

Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision

Hao Dong, Eleni Chatzi, Olga Fink

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV 2024, code: https://github.com/donghao51/MOOSA

2406.13898 2024-06-21 cs.CV cs.CL cs.CY 84%

The Use of Multimodal Large Language Models to Detect Objects from Thermal Images: Transportation Applications

Huthaifa I. Ashqar, Taqwa I. Alhadidi, Mohammed Elhenawy, Nour O. Khanfar

专题命中其他多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

2406.07078 2024-06-12 cs.CV cs.AI 84%

Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology

Huahui Yi, Xiaofei Wang, Kang Li, Chao Li

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2403.03385 2024-03-07 eess.IV cs.AI cs.CV 84%

Multi-modal Deep Learning

Chen Yuhua

专题命中其他多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Master's thesis

2402.02309 2024-02-07 cs.LG cs.CL cs.CR cs.CV 84%

Jailbreaking Attack against Multimodal Large Language Model

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, Rong Jin

专题命中其他多模态：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

2311.04257 2023-11-13 cs.CL cs.CV 84%

mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, Jingren Zhou

专题命中其他多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

2310.20357 2023-11-02 cs.AI cs.MM 84%

Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model

Yongqiang Zhao, Zhenyu Li, Zhi Jin, Feng Zhang, Haiyan Zhao, Chengfeng Dou, Zhengwei Tao, Xinhai Xu, Donghong Liu

专题命中其他多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.AI、cs.MM

2304.03086 2023-08-01 cs.CL cs.AI 84%

ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model

Hanyao Huang, Ou Zheng, Dongdong Wang, Jiayi Yin, Zijin Wang, Shengxuan Ding, Heng Yin, Chuan Xu, Renjie Yang, Qian Zheng, Bing Shi

专题命中其他多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Journal ref Int J Oral Sci. 2023 Jul 28;15(1):29

2303.09167 2023-03-17 cs.CV cs.AI 84%

Emotional Reaction Intensity Estimation Based on Multimodal Data

Shangfei Wang, Jiaqiang Wu, Feiyi Zheng, Xin Li, Xuewei Li, Suwen Wang, Yi Wu, Yanan Chang, Xiangyu Miao

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2211.15158 2023-03-16 cs.CV cs.AI cs.LG 84%

Heterogeneous Graph Learning for Multi-modal Medical Data Analysis

Sein Kim, Namkyeong Lee, Junseok Lee, Dongmin Hyun, Chanyoung Park

专题命中其他多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments AAAI 2023

2206.05555 2022-06-14 cs.CL cs.CV 84%

A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training

Zhihao Fan, Zhongyu Wei, Jingjing Chen, Siyuan Wang, Zejun Li, Jiarong Xu, Xuanjing Huang

专题命中其他多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

2101.11376 2021-09-06 cs.LG cs.AI cs.MM 84%

Learning Abstract Representations through Lossy Compression of Multi-Modal Signals

Charles Wilmot, Gianluca Baldassarre, Jochen Triesch

专题命中其他多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI、cs.MM

2010.10593 2021-02-03 cs.CV cs.AI 84%

Cross-Modal Information Maximization for Medical Imaging: CMIM

Tristan Sylvain, Francis Dutil, Tess Berthier, Lisa Di Jorio, Margaux Luck, Devon Hjelm, Yoshua Bengio

专题命中其他多模态：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICASSP 2021

1806.06176 2019-05-15 cs.LG cs.CL cs.CV stat.ML 84%

Learning Factorized Multimodal Representations

Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ICLR 2019

2307.03388 2023-07-10 cs.CV 84%

General-Purpose Multimodal Transformer meets Remote Sensing Semantic Segmentation

Nhi Kieu, Kien Nguyen, Sridha Sridharan, Clinton Fookes

专题命中其他多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted to CVPR Workshop on Multimodal Learning for Earth and Environment 2023

2003.07717 2020-07-09 cs.CV 84%

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Rundi Wu, Xuelin Chen, Yixin Zhuang, Baoquan Chen

专题命中其他多模态：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2020 (spotlight). Project page at https://chriswu1997.github.io/files/multimodal-pc/index.html

2608.10240 2026-08-12 cs.IR cs.LG cs.MM 新提交 83%

Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation

用于鲁棒多模态序列推荐的序列模态丢弃（Sequential Modality Dropout, SMD）

Guanqun Yang, Wenlong Zhang

专题命中其他多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.MM

AI总结本文提出序列模态丢弃（SMD），通过训练时随机擦除模态流提升多模态序列推荐器的鲁棒性，在四个骨干网络和亚马逊数据集上显著提升了缺失模态下的推荐准确率，且几乎不损失完整模态性能。

Comments Accepted at CIKM 2026. Code: https://github.com/guanqun-yang/SMD

1. 其他多模态 4884 篇

Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models

CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs

Speculative Decoding Reimagined for Multimodal Large Language Models

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models

Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration

Bridging Compressed Image Latents and Multimodal Large Language Models

Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models

Visual Grounding with Multi-modal Conditional Adaptation

VizECGNet: Visual ECG Image Network for Cardiovascular Diseases Classification with Multi-Modal Training and Knowledge Distillation

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model

DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM

Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision

The Use of Multimodal Large Language Models to Detect Objects from Thermal Images: Transportation Applications

Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology

Multi-modal Deep Learning

Jailbreaking Attack against Multimodal Large Language Model

mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model

ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model

Emotional Reaction Intensity Estimation Based on Multimodal Data

Heterogeneous Graph Learning for Multi-modal Medical Data Analysis

A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training

Learning Abstract Representations through Lossy Compression of Multi-Modal Signals

Cross-Modal Information Maximization for Medical Imaging: CMIM

Learning Factorized Multimodal Representations

General-Purpose Multimodal Transformer meets Remote Sensing Semantic Segmentation

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation