多模态大模型

Can Large Vision-Language Models Understand Multimodal Sarcasm?

Xinyu Wang, Yue Zhang, Liqiang Jing

机构 * The University of Texas at Dallas（德克萨斯大学达拉斯分校） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by CIKM 2025

Kestrel: 3D Multimodal LLM for Part-Aware Grounded Description

Mahmoud Ahmed, Junjie Fei, Jian Ding, Eslam Mohamed Bakr, Mohamed Elhoseiny

机构 * King Abdullah University of Science and Technology（卡斯特大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Ambiguity-Aware and High-Order Relation Learning for Multi-Grained Image-Text Matching

Junyu Chen, Yihua Gao, Mingyuan Ge, Mingyong Li

机构 * College of Computer and Information Science, Chongqing Normal University（重庆师范大学计算机与信息科学学院） ； School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件工程学院） 纠错

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted by the Knowledge-Based Systems(KBS), 2025

Journal ref Volume 316, 12 May 2025, 113355

SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities

Guoyang Xia, Yifeng Ding, Fengfa Li, Lei Ren, Wei Chen, Fangxiang Feng, Xiaojie Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMs

Zijian Zhang, Xuecheng Wu, Danlei Huang, Siyu Yan, Chong Peng, Xuezhi Cao

机构 * Xi'an Jiaotong University（西安交通大学） ； East China Normal University（华东师范大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models

Tianjie Ju, Yi Hua, Hao Fei, Zhenyu Shao, Yubin Zheng, Haodong Zhao, Mong-Li Lee, Wynne Hsu, Zhuosheng Zhang, Gongshen Liu

机构 * Shanghai Jiao Tong University（上海交通大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at ICML 2025

ProMedTS: A Self-Supervised, Prompt-Guided Multimodal Approach for Integrating Medical Text and Time Series

Shuai Niu, Jing Ma, Hongzhan Lin, Liang Bai, Zhihua Wang, Wei Bi, Yida Xu, Guo Li, Xian Yang

机构 * Hong Kong Baptist University（香港 Baptist 大学） ； Shanxi University（山西大学） ； Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海研究院） ； Tencent AI Lab（腾讯人工智能实验室） ； Manchester Metropolitan University（曼彻斯特 Metropolitan 大学） ； The University of Manchester（曼彻斯特大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments This paper is accepted by ACL2025(Findings)

Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning

Choi Changin, Lim Sungjun, Rhee Wonjong

机构 * Interdisciplinary Program in Artificial Intelligence（人工智能交叉学科项目） ； Department of Intelligence and Information（智能与信息系） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision

Lingxiao Du, Fanqing Meng, Zongkai Liu, Zhixiang Zhou, Ping Luo, Qiaosheng Zhang, Wenqi Shao

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models

Mingi Jung, Saehyung Lee, Eunji Kim, Sungroh Yoon

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ICML 2025

Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models

Yufei Zhan, Hongyin Zhao, Yousong Zhu, Shurong Zheng, Fan Yang, Ming Tang, Jinqiao Wang

机构 * Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室） ； Wuhan AI Research, Wuhan, China（武汉人工智能研究所） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Tech report

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation

Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark

机构 * University of Pennsylvania（宾夕法尼亚大学） ； Allen Institute for Artificial Intelligence（人工智能研究所） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published in ACL 2025, project page: https://yueyang1996.github.io/cosyn/

Debating for Better Reasoning: An Unsupervised Multimodal Approach

Ashutosh Adhikari, Mirella Lapata

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models

Mugilan Ganesan, Shane Segal, Ankur Aggarwal, Nish Sinnadurai, Sean Lie, Vithursan Thangarasa

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Main paper: 11 pages, 4 figures, 3 tables. Supplementary: 1 page

Vision-Language Models Create Cross-Modal Task Representations

Grace Luo, Trevor Darrell, Amir Bar

机构 * University of California, Berkeley, USA（加州大学伯克利分校） 纠错

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ICML 2025

CLIP-KOA: Enhancing Knee Osteoarthritis Diagnosis with Multi-Modal Learning and Symmetry-Aware Loss Functions

Yejin Jeong, Donghun Lee

机构 * Department of Mathematics, Korea University（数学系，韩国大学） 纠错

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures

DeepSPG: Exploring Deep Semantic Prior Guidance for Low-light Image Enhancement with Multimodal Learning

Jialang Lu, Huayu Zhao, Huiyu Zhai, Xingxing Yang, Shini Han

机构 * School of Cyber Science and Technology, Hubei University（湖北大学计算机科学与技术学院） ； Department of Electrical Automation Design, Beijing Shougang International Engineering Technology（北京首钢国际工程技术部） ； School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院） ； Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机科学部） ； School of Computer Science and Technology, Harbin University of Science and Technology（哈尔滨理工大学计算机科学与技术学院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICMR 2025 Main track. Code is available at https://github.com/Wenyuzhy/DeepSPG

FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model

Kaicheng Pang, Xingxing Zou, Waikeung Wong

机构 * Laboratory for Artificial Intelligence in Design（人工智能设计实验室） ； School of Fashion and Textiles（时尚与纺织学院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

Di Zhang, Junxian Li, Jingdi Lei, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, Peng Ye, Wanli Ouyang, Dongzhan Zhou

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Shanghai Jiaotong University（上海交通大学） ； Nankai University（南开大学） ； Shanghai University（上海大学） ； Nanyang Technological University（南洋理工大学） ； Hong Kong Polytechnic University（香港理工大学） ； Tongji University（同济大学） ； Nanjing University（南京大学） ； University of California, Merced（加州大学梅德福分校） ； Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 16 pages, 11 figures

Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models

Chen Wang, Fei Xia, Wenhao Yu, Tingnan Zhang, Ruohan Zhang, C. Karen Liu, Li Fei-Fei, Jie Tan, Jacky Liang

机构 * Google DeepMind（谷歌DeepMind） ； Stanford University（斯坦福大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments ICRA 2025

Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models

Yuxiang Lin, Jingdong Sun, Zhi-Qi Cheng, Jue Wang, Haomin Liang, Zebang Cheng, Yifei Dong, Jun-Yan He, Xiaojiang Peng, Xian-Sheng Hua

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted at CVPR Workshop NEXD 2025. 21 pages, Project: https://github.com/Lum1104/EIBench

Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection

Ye Jiang, Yimin Wang

专题命中 图文多模态 ：multimodal(title)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments Withdraw for new experiments

CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections

Florian Schneider, Narges Baba Ahmadi, Niloufar Baba Ahmadi, Iris Vogel, Martin Semmann, Chris Biemann

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

Yuandong Pu, Le Zhuo, Kaiwen Zhu, Liangbin Xie, Wenlong Zhang, Xiangyu Chen, Peng Gao, Yu Qiao, Chao Dong, Yihao Liu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

SmolVLM: Redefining small and efficient multimodal models

Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, Thomas Wolf

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Vision-Language Models for Acute Tuberculosis Diagnosis: A Multimodal Approach Combining Imaging and Clinical Data

Ananya Ganapthy, Praveen Shastry, Naveen Kumarasami, Anandakumar D, Keerthana R, Mounigasri M, Varshinipriya M, Kishore Prasath Venkatesh, Bargava Subramanian, Kalyan Sivasailam

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 3 figures

HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding

Rui Yang, Lin Song, Yicheng Xiao, Runhui Huang, Yixiao Ge, Ying Shan, Hengshuang Zhao

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries

Tianle Li, Yongming Rao, Winston Hu, Yu Cheng

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning

Yanjun Chen, Yirong Sun, Xinghao Chen, Jian Wang, Xiaoyu Shen, Wenjie Li, Wei Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioning

Qifeng Zhou, Wenliang Zhong, Yuzhi Guo, Michael Xiao, Hehuan Ma, Junzhou Huang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI