多模态大模型

Hierarchical Vision-Language Reasoning for Multimodal Multiple-Choice Question Answering

Ao Zhou, Zebo Gu, Tenghao Sun, Jiawen Chen, Mingsheng Tu, Zifeng Cheng, Yafeng Yin, Zhiwei Jiang, Qing Gu

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学） ； Chongqing University of Posts and Telecommunications（重庆邮电大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments This paper has been accepted by ACM MM 2025

Interpreting the linear structure of vision-language model embedding spaces

Isabel Papadimitriou, Huangyuan Su, Thomas Fel, Sham Kakade, Stephanie Gil

机构 * Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University（哈佛大学自然与人工智能研究所） ； Department of Computer Science, Harvard University（哈佛大学计算机科学系） 纠错

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments COLM 2025

A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer

Yuhui Tao, Zhongwei Zhao, Zilong Wang, Xufang Luo, Feng Chen, Kang Wang, Chuanfu Wu, Xue Zhang, Shaoting Zhang, Jiaxi Yao, Xingwei Jin, Xinyang Jiang, Yifan Yang, Dongsheng Li, Lili Qiu, Zhiqiang Shao, Jianming Guo, Nengwang Yu, Shuo Wang, Ying Xiong

机构 * Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, 200032, China（复旦大学基础医学学院数字医学研究中心） ； Shanghai Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention, Shanghai, 200032, China（上海市医疗影像计算与计算机辅助干预重点实验室） ； Department of Urology, Qilu Hospital of Shandong University, Jinan, Shandong, 250012, China（山东大学齐鲁医院泌尿科） ； Microsoft Research Asia, Shanghai, 200232, China（微软亚洲研究院） ； Department of Radiology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310006, China（浙江大学医学院附属第一医院放射科） ； Center of Health data science, Linyi People’s Hospital, Shandong, 276003, China（临沂人民医院健康数据科学中心） ； Shandong Open Laboratory of Data Innovation Application, Shandong, 276003, China（山东省数据创新应用开放实验室） ； Department of Radiology, the First People’s Hospital of Lianyungang, Lianyungang, 222002, China（连云港第一人民医院放射科） ； Department of Urology, Zhangye People’s Hospital affiliated to Hexi University, Zhangye, 734000, China（张掖人民医院（河西大学附属）泌尿科） ； Department of Urology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China（上海交通大学附属瑞金医院泌尿科） ； Department of Urology, Linyi People’s Hospital, Shandong, 276003, China（临沂人民医院泌尿科） ； Department of Urology, Zhongshan Hospital, Fudan University, Shanghai, 200032, China（复旦大学中山医院泌尿科） 纠错

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Models

Xinyu Chen, Haotian Zhai, Can Zhang, Xiupeng Shi, Ruirui Li

机构 * Shanghai University（上海大学） ； Beijing University of Chemical Technology（北京化工大学） ； University of Minnesota（明尼苏达大学） 纠错

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICCV 2025

Do What? Teaching Vision-Language-Action Models to Reject the Impossible

Wen-Han Hsieh, Elvis Hsieh, Dantong Niu, Trevor Darrell, Roei Herzig, David M. Chan

机构 * University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.AI

Comments 9 pages, 2 figures, 1 table

Take That for Me: Multimodal Exophora Resolution with Interactive Questioning for Ambiguous Out-of-View Instructions

Akira Oyama, Shoichi Hasegawa, Akira Taniguchi, Yoshinobu Hagiwara, Tadahiro Taniguchi

机构 * Ritsumeikan University（立命馆大学） ； Soka University（早稻田大学） ； Kyoto University（京都大学） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments See website at https://emergentsystemlabstudent.github.io/MIEL/. Accepted at IEEE RO-MAN 2025

AI-Supported Mini-Labs: Combining Smartphone-Based Experiments and Multimodal AI

Jochen Kuhn, David J. Rakestraw, Stefan Küchemann, Patrik Vogt

专题命中 音频语音多模态 ：multimodal(title,abstract)

Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models

Nouar AlDahoul, Yasir Zaki

机构 * Computer Science Department, New York University Abu Dhabi（纽约大学阿布扎克分校计算机科学系） 纠错

专题命中 音频语音多模态 ：multi-modal(title)；分类 cs.CL、cs.AI

Comments 26 pages, 12 figures

Foundation Models for Cross-Domain EEG Analysis Application: A Survey

Hongqi Li, Yitong Chen, Yujuan Wang, Weihang Ni, Haodong Zhang

机构 * School of Software, Northwestern Polytechnical University（软件学院，西北工业大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.AI

Comments Submitted to IEEE Journals

Sentiment Reasoning for Healthcare

Khai-Nguyen Nguyen, Khai Le-Duc, Bach Phan Tat, Duy Le, Long Vo-Dang, Truong-Son Hy

机构 * College of William and Mary（威廉与玛丽学院） ； University of Toronto（多伦多大学） ； University Health Network（大学健康网络） ； KU Leuven（鲁汶大学） ； Bucknell University（巴克内尔大学） ； University of Cincinnati（辛辛那提大学） ； University of Alabama at Birmingham（伯明翰大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI、eess.AS

Comments ACL 2025 Industry Track (Oral)

Evaluating Speech-to-Text x LLM x Text-to-Speech Combinations for AI Interview Systems

Rumi Allbert, Nima Yazdani, Ali Ansari, Aruj Mahajan, Amirhossein Afsharrad, Seyed Shahabeddin Mousavi

机构 * University of Southern California（南加州大学） ； Stanford University（斯坦福大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Visuo-Acoustic Hand Pose and Contact Estimation

Yuemin Mao, Uksang Yoo, Yunchao Yao, Shahram Najam Syed, Luca Bondi, Jonathan Francis, Jean Oh, Jeffrey Ichnowski

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Learning Long-Range Action Representation by Two-Stream Mamba Pyramid Network for Figure Skating Assessment

Fengshun Wang, Qiurui Wang, Peilin Zhao

机构 * Capital University of \ Education ； Shanghai Jiao Tong University Shanghai China ； Shanghai Jiao Tong University 纠错

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM

Spacetime-GR: A Spacetime-Aware Generative Model for Large Scale Online POI Recommendation

Haitao Lin, Zhen Yang, Jiawei Xue, Ziji Zhang, Luzhu Wang, Yikun Gu, Yao Xu, Xin Li

机构 * AMAP, Alibaba Group（阿里集团AMAP） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI

VT-LVLM-AR: A Video-Temporal Large Vision-Language Model Adapter for Fine-Grained Action Recognition in Long-Term Videos

Kaining Li, Shuwei He, Zihan Xu

机构 * Xidian University（西电大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Order-Preserving Dimension Reduction for Multimodal Semantic Embedding

Chengyu Gong, Gefei Shen, Luanzheng Guo, Nathan Tallent, Dongfang Zhao

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Contributions to Label-Efficient Learning in Computer Vision and Remote Sensing

Minh-Tan Pham

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Habilitation à Diriger des Recherches (HDR) manuscript

MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds

Bingquan Dai, Li Ray Luo, Qihong Tang, Jie Wang, Xinyu Lian, Hao Xu, Minghan Qin, Xudong Xu, Bo Dai, Haoqian Wang, Zhaoyang Lyu, Jiangmiao Pang

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Tsinghua University（清华大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Beijing Institute of Technology（北京理工大学） ； AI Thrust, HKUST(GZ)（AI thrust, 香港科技大学（广州）） 纠错

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding

Mohan Jiang, Jin Gao, Jiahao Zhan, Dequan Wang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai Innovation Institute（上海创新研究院） ； Fudan University（复旦大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；cross-modal(abstract)；image-text(abstract)

ComicScene154: A Scene Dataset for Comic Analysis

Sandro Paval, Ivan P. Yamshchikov, Pascal Meißner

专题命中 多模态评测 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CL

XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning

Zhihan Zhang, Yixin Cao, Lizi Liao

机构 * School of Computing and Information Systems, Singapore Management University（计算与信息系统学院，新加坡管理大学） ； Institute of Trustworthy Embodied AI, Fudan University（可信具身人工智能研究院，复旦大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CL

Boosting Pathology Foundation Models via Few-shot Prompt-tuning for Rare Cancer Subtyping

Dexuan He, Xiao Zhou, Wenbin Guan, Liyuan Zhang, Xiaoman Zhang, Sinuo Xu, Ge Wang, Lifeng Wang, Xiaojun Yuan, Xin Sun, Yanfeng Wang, Kun Sun, Ya Zhang, Weidi Xie

机构 * School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院） ； Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系） ； Department of Oral Pathology, Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院第九人民医院口腔病学部） ； Department of Pathology, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院新华医院病理科） ； Department of Pediatric Hematology/Oncology, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院新华医院儿童血液肿瘤科） ； Clinical Research and Innovation Unit, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院新华医院临床研究与创新单元） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 多模态评测 ：cross-modal(abstract)；分类 cs.CV

AudioSet-R: A Refined AudioSet with Multi-Stage LLM Label Reannotation

Yulin Sun, Qisheng Xu, Yi Su, Qian Zhu, Yong Dou, Xinwang Liu, Kele Xu

机构 * College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学） 纠错

专题命中 多模态评测 ：cross-modal(abstract)

Comments 8 pages, 5 figures, accepted in ACM MM 2025 dataset track

DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking

Fang Wang, Tianwei Yan, Zonghao Yang, Minghao Hu, Jun Zhang, Zhunchen Luo, Xiaoying Bai

机构 * School of Computer Science, Peking University（北京大学计算机科学学院） ； School of Information Science and Engineering, Chongqing Jiaotong University（重庆交通大学信息科学与工程学院） ； China Research and Development Academy of Machinery Equipment（机械电子研究发展院） ； Center of Information Research, Academy of Military Science（军事科学信息研究中心） 纠错

专题命中 多模态Agent ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

A Multimodal-Multitask Framework with Cross-modal Relation and Hierarchical Interactive Attention for Semantic Comprehension

Mohammad Zia Ur Rehman, Devraj Raghuvanshi, Umang Jain, Shubhi Bansal, Nagendra Kumar

机构 * Indian Institute of Technology Indore（印度印度理工学院Indore） ； Brown University（布朗大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Published in Information Fusion

MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction

Ziyang Yan, Ruikai Li, Zhiyong Cui, Bohan Li, Han Jiang, Yilong Ren, Aoyong Li, Zhenning Li, Sijia Wen, Haiyang Yu

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Cross-Modal Prototype Augmentation and Dual-Grained Prompt Learning for Social Media Popularity Prediction

Ao Zhou, Mingsheng Tu, Luping Wang, Tenghao Sun, Zifeng Cheng, Yafeng Yin, Zhiwei Jiang, Qing Gu

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；multimodal(abstract)

Comments This paper has been accepted by ACM MM 2025

FlexMUSE: Multimodal Unification and Semantics Enhancement Framework with Flexible interaction for Creative Writing

Jiahao Chen, Zhiyong Ma, Wenbiao Du, Qingyuan Chuai

机构 * Cao Tu Li(Guangzhou) Technology Co., Ltd, China（广东曹图利技术有限公司） ； South China University of Technology, China（华南理工大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs

Yijie Jin, Junjie Peng, Xuanchao Lin, Haochen Yuan, Lan Wang, Cangzhi Zheng

机构 * School of Computer Engineering and Science, Shanghai University（上海大学计算机工程与科学学院） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref https://aclanthology.org/2025.acl-long.109/

SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather

Edoardo Palladin, Roland Dietze, Praveen Narayanan, Mario Bijelic, Felix Heide

机构 * Torc Robotics（Torc机器人公司） ； University of Stuttgart（斯图加特大学） ； Princeton University（普林斯顿大学） 纠错

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV