多模态大模型

RadGPT: Constructing 3D Image-Text Tumor Datasets

Pedro R. A. S. Bassi, Mehmet Can Yavuz, Kang Wang, Xiaoxi Chen, Wenxuan Li, Sergio Decherchi, Andrea Cavalli, Yang Yang, Alan Yuille, Zongwei Zhou

机构 * Johns Hopkins University（约翰霍普金斯大学） ； University of Bologna（博洛尼亚大学） ； Italian Institute of Technology（意大利理工学院） ； University of California, San Francisco（加州大学旧金山分校） ； Istanbul Medipol University（伊斯坦布尔Medipol大学） ； University of Zurich（苏黎世大学） ； ETH AI Center（ETH人工智能中心） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院） 纠错

专题命中 图文多模态 ：image-text(title)；分类 cs.CV

MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence

Sonal Kumar, Šimon Sedláček, Vaibhavi Lokegaonkar, Fernando López, Wenyi Yu, Nishit Anand, Hyeonggon Ryu, Lichang Chen, Maxim Plička, Miroslav Hlaváček, William Fineas Ellingwood, Sathvik Udupa, Siyuan Hou, Allison Ferner, Sara Barahona, Cecilia Bolaños, Satish Rahi, Laura Herrera-Alarcón, Satvik Dixit, Siddhi Patil, Soham Deshmukh, Lasha Koroshinadze, Yao Liu, Leibny Paola Garcia Perera, Eleni Zanou, Themos Stafylakis, Joon Son Chung, David Harwath, Chao Zhang, Dinesh Manocha, Alicia Lozano-Diez, Santosh Kesiraju, Sreyan Ghosh, Ramani Duraiswami

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Can Large Language Models (LLMs) Describe Pictures Like Children? A Comparative Corpus Study

Hanna Woloszyn, Benjamin Gagl

机构 * Self-Learning Systems Lab, Faculty of Human Sciences, Department of Special Education and Rehabilitation（自主学习系统实验室，人文科学学院，特殊教育与康复系） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes

Keliang Li, Hongze Shen, Hao Shi, Ruibing Hou, Hong Chang, Jie Huang, Chenghao Jia, Wen Wang, Yiling Wu, Dongmei Jiang, Shiguang Shan, Xilin Chen

机构 * Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, China（中国科学院智能信息处理重点实验室（中国科学院）、计算技术研究所、中国） 纠错

专题命中 视频多模态 ：MLLM(title)；multimodal(abstract)；分类 cs.CV

Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction

Zeren Jiang, Chuanxia Zheng, Iro Laina, Diane Larlus, Andrea Vedaldi

机构 * Visual Geometry Group, University of Oxford（视觉几何组，牛津大学） ； Naver Labs Europe（Naver欧洲实验室） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 17 pages, 6 figures, ICCV 2025 Highlight, Project page: https://geo4d.github.io/

Spatially-guided Temporal Aggregation for Robust Event-RGB Optical Flow Estimation

Qianang Zhou, Junhui Hou, Meiyi Yang, Yongjian Deng, Youfu Li, Junlin Xiong

机构 * Department of Automation, University of Science and Technology of China（自动化系，中国科学技术大学） ； Department of Computer Science, City University of Hong Kong（计算机科学系，香港城市大学） ； Department of Mechanical Engineering, City University of Hong Kong（机械工程系，香港城市大学） ； College of Computer Science, Beijing University of Technology（计算机科学学院，北京理工大学） 纠错

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Comments 11 pages, 8 figures, under review

YOLO11-CR: a Lightweight Convolution-and-Attention Framework for Accurate Fatigue Driving Detection

Zhebin Jin, Ligang Dong

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Multimodal Data Storage and Retrieval for Embodied AI: A Survey

Yihao Lu, Hao Tang

机构 * School of Economics and Management, South China Normal University（经济管理学院，华南师范大学） ； State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMs

Yi-Ting Shen, Sungmin Eum, Doheon Lee, Rohit Shete, Chiao-Yi Wang, Heesung Kwon, Shuvra S. Bhattacharyya

机构 * University of Maryland, College Park（马里兰大学学院市分校） ； DEVCOM Army Research Laboratory（陆军研究实验室） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments ICCV 2025

Refining Contrastive Learning and Homography Relations for Multi-Modal Recommendation

Shouxing Ma, Yawen Zeng, Shiqing Wu, Guandong Xu

专题命中 跨模态检索 ：multi-modal(title,abstract)

Comments This paper has been accepted as a full paper at ACM MM 2025

Towards Unified Multimodal Financial Forecasting: Integrating Sentiment Embeddings and Market Indicators via Cross-Modal Attention

Sarthak Khanna, Armin Berger, David Berghaus, Tobias Deusser, Lorenz Sparrenberg, Rafet Sifa

机构 * Fraunhofer IAIS - Department of Media Engineering（弗劳恩霍夫研究所媒体工程部门） ； University of Bonn - Department of Computer Science（波恩大学计算机科学系） ； West-AI - Federal Ministry of Education and Research（西德人工智能 - 教育与研究部） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.AI

Comments Accepted in IEEE-DSAA 2025

A Fully Transformer Based Multimodal Framework for Explainable Cancer Image Segmentation Using Radiology Reports

Enobong Adahada, Isabel Sassoon, Kate Hone, Yongmin Li

机构 * Department of Computer Science（计算机科学系） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

Zhe Xu, Ziyi Liu, Junlin Hou, Jiabo Ma, Cheng Jin, Yihui Wang, Zhixuan Chen, Zhengyu Zhang, Fuxiang Huang, Zhengrui Guo, Fengtao Zhou, Yingxue Xu, Xi Wang, Ronald Cheong Kin Chan, Li Liang, Hao Chen

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Southern Medical University（南方医科大学） ； Nanfang Hospital and School of Basic Medical Sciences（南方医院和基础医学科学学院） ； Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Benchmarking GPT-5 for Zero-Shot Multimodal Medical Reasoning in Radiology and Radiation Oncology

Mingzhe Hu, Zach Eidex, Shansong Wang, Mojtaba Safari, Qiang Li, Xiaofeng Yang

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

Shilong Li, Xingyuan Bu, Wenjie Wang, Jiaheng Liu, Jun Dong, Haoyang He, Hao Lu, Haozhe Zhang, Chenchen Jing, Zhen Li, Chuanhao Li, Jiayi Tian, Chenchen Zhang, Tianhao Peng, Yancheng He, Jihao Gu, Yuanxing Zhang, Jian Yang, Ge Zhang, Wenhao Huang, Wangchunshu Zhou, Zhaoxiang Zhang, Ruizhe Ding, Shilei Wen

机构 * Nanjing University（南京大学） ； Zhejiang University（浙江大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments The first two authors contribute equally, 26 pages, repo at https://github.com/MMBrowseComp/MM-BrowseComp

MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models

Jiacheng Ruan, Dan Jiang, Xian Gao, Ting Liu, Yuzhuo Fu, Yangyang Kang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 9 pages, 6 figures, work in progress

CORENet: Cross-Modal 4D Radar Denoising Network with LiDAR Supervision for Autonomous Driving

Fuyang Liu, Jilin Mei, Fangyuan Mao, Chen Min, Yan Xing, Yu Hu

机构 * Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Beijing Institute of Control Engineering（北京控制工程研究所） 纠错

专题命中 多模态评测 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 5 figures, Accepted to IROS 2025

Early Detection of Pancreatic Cancer Using Multimodal Learning on Electronic Health Records

Mosbah Aouad, Anirudh Choudhary, Awais Farooq, Steven Nevers, Lusine Demirkhanyan, Bhrandon Harris, Suguna Pappu, Christopher Gondi, Ravishankar Iyer

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； University of Illinois Chicago（伊利诺伊大学芝加哥分校） ； University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Proceedings of Machine Learning for Healthcare (2025)

Cross-Modal Characterization of Thin Film MoS$_2$ Using Generative Models

Isaiah A. Moses, Chen Chen, Joan M. Redwing, Wesley F. Reinhart

机构 * Materials Research Institute, The Pennsylvania State University（材料研究学院，宾夕法尼亚州立大学） ； Department of Materials Science and Engineering, The Pennsylvania State University（材料科学与工程系，宾夕法尼亚州立大学） ； Institute for Computational and Data Sciences, The Pennsylvania State University（计算与数据科学研究院，宾夕法尼亚州立大学） 纠错

专题命中 多模态评测 ：cross-modal(title,abstract)

Comments 45 pages, 11 figures, 12 tables

MMIS-Net for Retinal Fluid Segmentation and Detection

Nchongmaje Ndipenocha, Alina Mirona, Kezhi Wanga, Yongmin Li

机构 * Brunel University London（布鲁内尔大学伦敦分校） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV

Age-Normalized HRV Features for Non-Invasive Glucose Prediction: A Pilot Sleep-Aware Machine Learning Study

Md Basit Azam, Sarangthem Ibotombi Singh

机构 * Department of Computer Science & Engineering（计算机科学与工程系） ； Tezpur University（泰浦大学） 纠错

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.AI

Categorical Policies: Multimodal Policy Learning and Exploration in Continuous Control

SM Mazharul Islam, Manfred Huber

机构 * Department of Computer Science and Engineering, University of Texas at Arlington（计算机科学与工程系，德克萨斯大学阿灵顿分校） 纠错

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments 6 pages, 4 figures; Has been submitted and accepted at IEEE SMC, 2025

Integrating emotional intelligence, memory architecture, and gestures to achieve empathetic humanoid robot interaction in an educational setting

Fuze Sun, Lingyu Li, Shixiangyue Meng, Xiaoming Teng, Terry R. Payne, Paul Craig

机构 * University of Liverpool（利物浦大学） ； Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学） 纠错

专题命中 多模态Agent ：multi-modal(abstract)

On-Orbit Servicing Optimization Framework with High- and Low-Thrust Propulsion Tradeoff

Tristan Sarton du Jonchay, Hao Chen, Masafumi Isaji, Yuri Shimane, Koki Ho

专题命中 多模态Agent ：multimodal(abstract)

Comments 45 pages, 16 figures, Accepted by and Published in the Journal of Spacecraft and Rockets (Accepted Version)

Journal ref Journal of Spacecraft and Rockets, Volume 59, Number 1, January 2022

UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion

Zihan Liang, Yufei Ma, ZhiPeng Qian, Huangyu Dai, Zihan Wang, Ben Chen, Chenyi Lei, Yuqing Ding, Han Li

机构 * Kuaishou Technology（快手科技） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.AI

Comments Accepted at CIKM2025 as a long paper

SPANER: Shared Prompt Aligner for Multimodal Semantic Representation

Thye Shan Ng, Caren Soyeon Han, Eun-Jung Holden

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(abstract)；audio-visual(abstract)；分类 cs.AI

Revisiting MLLM Token Technology through the Lens of Classical Visual Coding

Jinming Liu, Junyan Lin, Yuntao Wei, Kele Shao, Keda Tao, Jianguo Huang, Xudong Yang, Zhibo Chen, Huan Wang, Xin Jin

机构 * Eastern Institute of Technology, Ningbo, China（东部技术研究所） ； Westlake University（西湖大学） ； USTC（中国科学技术大学） 纠错

专题命中 多模态训练与对齐 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience

Andrew Chang, Chenkai Hu, Ji Qi, Zhuojian Wei, Kexin Zhang, Viswadruth Akkaraju, David Poeppel, Dustin Freeman

机构 * New York UniversityUSA（纽约大学） ； Max Planck SocietyGermany（马克斯·普朗克研究所） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.MM、eess.AS

Comments Interspeech 2025

MAGNeT: Multimodal Adaptive Gaussian Networks for Intent Inference in Moving Target Selection across Complex Scenarios

Xiangxian Li, Yawen Zheng, Baiqiao Zhang, Yijia Ma, Xianhui Cao, Juan Liu, Yulong Bian, Jin Huang, Chenglei Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.MM

Comments Accepted by ACM MM 2025

Contextual Attention-Based Multimodal Fusion of LLM and CNN for Sentiment Analysis

Meriem Zerkouk, Miloud Mihoubi, Belkacem Chikhaoui

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments The 38th Canadian Conference on Artificial Intelligence ( 2025 )