多模态大模型

Audio-Enhanced Vision-Language Modeling with Latent Space Broadening for High Quality Data Expansion

Yu Sun, Yin Li, Ruixiao Sun, Chunhui Liu, Fangming Zhou, Ze Jin, Linjie Wang, Xiang Shen, Zhuolin Hao, Hongyu Xiong

机构 * ByteDance Inc.（字节跳动公司） 纠错

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoning

Xuchen Li, Xuzhao Li, Jiahui Gao, Renjie Pi, Shiyu Hu, Wentao Zhang

机构 * CASIA（中国科学院自动化研究所） ； UCAS（中国科学技术大学） ； ZGCA（北京智感科技有限公司） ； HKU（香港大学） ； HKUST（香港科技大学） ； NTU（国立台湾大学） ； PKU（北京大学） 纠错

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Preprint, Under review

Towards fairer public transit: Real-time tensor-based multimodal fare evasion and fraud detection

Peter Wauyo, Dalia Bwiza, Alain Murara, Edwin Mugume, Eric Umuhoza

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)

Comments 10 pages

Multimodal Feedback for Task Guidance in Augmented Reality

Hu Guo, Lily Patel, Rohan Gupt

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)

NeRAF: 3D Scene Infused Neural Radiance and Acoustic Fields

Amandine Brunetto, Sascha Hornauer, Fabien Moutarde

机构 * Center for Robotics, Mines Paris - PSL University Paris, France（机器人中心，巴黎 Mines Paris - PSL 大学） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；audio-visual(abstract)；分类 cs.CV、eess.AS

Comments ICLR 2025 (Poster). Camera ready version. Project Page: https://amandinebtto.github.io/NeRAF; 24 pages, 13 figures

Journal ref The Thirteenth International Conference on Learning Representations, 2025

Clink! Chop! Thud! -- Learning Object Sounds from Real-World Interactions

Mengyu Yang, Yiming Chen, Haozheng Pei, Siddhant Agarwal, Arun Balajee Vasudevan, James Hays

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments ICCV 2025. Project page: https://clink-chop-thud.github.io/

SoundReactor: Frame-level Online Video-to-Audio Generation

Koichi Saito, Julian Tanke, Christian Simon, Masato Ishii, Kazuki Shimada, Zachary Novack, Zhi Zhong, Akio Hayakawa, Takashi Shibuya, Yuki Mitsufuji

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook

Haoxin Liu, Harshavardhan Kamarthi, Zhiyuan Zhao, Shangqing Xu, Shiyu Wang, Qingsong Wen, Tom Hartvigsen, Fei Wang, B. Aditya Prakash

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Bytedance Inc.（字节跳动公司） ； Squirrel AI, USA（squirrel AI 美国分公司） ； The University of Virginia（弗吉尼亚大学） ； Cornell University（康奈尔大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Github Repo: https://github.com/AdityaLab/MM4TSA Updated to include papers accepted by IJCAI25, KDD25, ICML25, NeurIPS25 4 figures or tables, 19 pages, 251 references

Poolformer: Recurrent Networks with Pooling for Long-Sequence Modeling

Daniel Gallo Fernández

机构 * MSc Artificial Intelligence Master Thesis（人工智能硕士论文） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)

From Videos to Indexed Knowledge Graphs -- Framework to Marry Methods for Multimodal Content Analysis and Understanding

Basem Rizk, Joel Walsh, Mark Core, Benjamin Nye

机构 * University Of Southern California（美国南加州大学） 纠错

专题命中 视频多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

MultiModal Action Conditioned Video Generation

Yichen Li, Antonio Torralba

机构 * MIT CSAIL（麻省理工学院计算机科学与人工智能实验室） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

LLM4Rec: Large Language Models for Multimodal Generative Recommendation with Causal Debiasing

Bo Ma, Hang Li, ZeHua Hu, XiaoFan Gui, LuYao Liu, Simon Lau

机构 * Department of Software \& Microelectronics Peking University Beijing, China ； Department of Software \& Microelectronics Peking University Beijing, China hangli\ ； Department of Software \& Microelectronics Peking University Beijing, China zehua\ ； Department of Software \& Microelectronics Peking University Beijing, China xiaofan\ ； Economic Law School China University of Political Science ； Peking University Changsha, China 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

CardioRAG: A Retrieval-Augmented Generation Framework for Multimodal Chagas Disease Detection

Zhengyang Shen, Xuehao Zhai, Hua Tu, Mayue Shi

机构 * Department of Electrical and Electronic Engineering Imperial College London（帝国理工学院电子与电气工程系） ； Department of Civil and Environmental Engineering Imperial College London（帝国理工学院土木与环境工程系） ； Institute of Biomedical Engineering Department of Engineering Science University of Oxford（牛津大学生物医学工程研究所） 纠错

专题命中 跨模态检索 ：multimodal(title)

Comments 4 pages, 2 figures. Accepted for oral presentation at the 52nd international Computing in Cardiology Conference (CinC2025)

Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation

Chetwin Low, Weimin Wang, Calder Katyal

机构 * Character AI ； Yale University（耶鲁大学） 纠错

专题命中 多模态生成 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM、eess.AS

Design and Application of Multimodal Large Language Model Based System for End to End Automation of Accident Dataset Generation

MD Thamed Bin Zaman Chowdhury, Moazzem Hossain

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CL

Comments This paper is accepted for presentation in TRB annual meeting 2026. The version presented here is the preprint version before peer review process

VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation

Xuanran Zhai, Qianyou Zhao, Qiaojun Yu, Ce Hao

机构 * National University of Singapore（新加坡国立大学） ； Shanghai Jiao Tong University（上海交通大学） ； Shanghai AI Lab（上海人工智能实验室） 纠错

专题命中 多模态生成 ：multi-modal(title)

Enhanced DACER Algorithm with High Diffusion Efficiency

Yinuo Wang, Likun Wang, Mining Tan, Wenjun Zou, Xujie Song, Wenxuan Wang, Tong Liu, Guojian Zhan, Tianze Zhu, Shiqi Liu, Zeyu He, Feihong Zhang, Jingliang Duan, Shengbo Eben Li

机构 * School of Vehicle and Mobility & College of AI, Tsinghua University（车辆与移动学院及人工智能学院，清华大学） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学） ； School of Mechanical Engineering, University of Science and Technology Beijing（机械工程学院，北京科技大学） 纠错

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.AI

Handling Heterophily in Recommender Systems with Wavelet Hypergraph Diffusion

Darnbi Sakong, Thanh Tam Nguyen

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

Comments Fixed and extended results

DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cut

Paul Couairon, Mustafa Shukor, Jean-Emmanuel Haugeard, Matthieu Cord, Nicolas Thome

专题命中 多模态生成 ：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024. Project page at https://diffcut-segmentation.github.io. Code at https://github.com/PaulCouairon/DiffCut

SlideItRight: Using AI to Find Relevant Slides and Provide Feedback for Open-Ended Questions

Chloe Qianhui Zhao, Jie Cao, Eason Chen, Kenneth R. Koedinger, Jionghao Lin

专题命中 多模态生成 ：multimodal(abstract)

Comments 14 pages, to be published at the 26th International Conference on Artificial Intelligence in Education (AIED '25)

Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretations

Ricardo Gonzalez Penuela, Felipe Arias-Russi, Victor Capriles

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 2 figure, 2 tables, CV4A11y Workshop at ICCV 2025

MDSEval: A Meta-Evaluation Benchmark for Multimodal Dialogue Summarization

Yinhong Liu, Jianfeng He, Hang Su, Ruixue Lian, Yi Nian, Jake Vincent, Srikanth Vishnubhotla, Robinson Piramuthu, Saab Mansour

机构 * AWS AI Labs（AWS人工智能实验室） ； Language Technology Lab, University of Cambridge（语言技术实验室，剑桥大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2025

What are You Looking at? Modality Contribution in Multimodal Medical Deep Learning

Christian Gapp, Elias Tappeiner, Martin Welk, Karl Fritscher, Elke Ruth Gizewski, Rainer Schubert

机构 * Institute of Biomedical Image Analysis UMIT TIROL -- Private University for Health Sciences（生物医学影像分析研究所） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Contribution to Conference for Computer Assisted Radiology and Surgery (CARS 2025)

Journal ref Int J CARS (2025)

ImageNet-Think-250K: A Large-Scale Synthetic Dataset for Multimodal Reasoning for Vision Language Models

Krishna Teja Chitty-Venkata, Murali Emani

机构 * Argonne National Laboratory（阿贡国家实验室） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Preprint

A modular framework for collaborative human-AI, multi-modal and multi-beamline synchrotron experiments

Adam A. Corrao, Phillip M. Maffettone, Bruce Ravel, Thomas A. Caswell, Stuart I. Campbell, Howie Joress, Stuart Wilkins, Daniel Olds

专题命中 多模态评测 ：multi-modal(title,abstract)

Comments 39 pages, 13 figures, Supporting Information available in SI.pdf

EyePCR: A Comprehensive Benchmark for Fine-Grained Perception, Knowledge Comprehension and Clinical Reasoning in Ophthalmic Surgery

Gui Wang, Yang Wennuo, Xusen Ma, Zehao Zhong, Zhuoru Wu, Ende Wu, Rong Qu, Wooi Ping Cheah, Jianfeng Ren, Linlin Shen

机构 * School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院） ； School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波校区计算机科学学院） ； Wenzhou Medical University（温州医科大学） ； School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院） 纠错

专题命中 多模态评测 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Strong accept by NeurIPS2025 Reviewers and AC

From 2D to 3D, Deep Learning-based Shape Reconstruction in Magnetic Resonance Imaging: A Review

Emma McMillian, Abhirup Banerjee, Alfonso Bueno-Orovio

机构 * University of Oxford（牛津大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CV、cs.AI

Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports

Punit Kumar Singh, Nishant Kumar, Akash Ghosh, Kunal Pasad, Khushi Soni, Manisha Jaishwal, Sriparna Saha, Syukron Abu Ishaq Alfarozi, Asres Temam Abagissa, Kitsuchart Pasupa, Haiqin Yang, Jose G Moreno

机构 * Indian Institute of Technology Patna（印度理工学院帕纳布分校） ； Sardar Patel Institute of Technology（萨达尔·帕特尔技术学院） ； Universitas Gadjah Mada（加查马大学） ； King Mongkut’s Institute of Technology Ladkrabang（拉差班国王技术学院） ； Shenzhen Technology University（深圳技术大学） ； Université de Toulouse（图卢兹大学） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 52 pages, 56 figures; appearing at EMNLP'25

MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMs

Jiyao Liu, Jinjie Wei, Wanying Qu, Chenglong Ma, Junzhi Ning, Yunheng Li, Ying Chen, Xinzhe Luo, Pengcheng Chen, Xin Gao, Ming Hu, Huihui Xu, Xin Wang, Shujian Gao, Dingkang Yang, Zhongying Deng, Jin Ye, Lihao Liu, Junjun He, Ningsheng Xu

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Imperial College London（帝国理工学院） ； University of Cambridge（剑桥大学） 纠错

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV

Comments 26 pages, 13 figures

Latency-aware Multimodal Federated Learning over UAV Networks

Shaba Shaon, Dinh C. Nguyen

机构 * ECE Department, University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校电子与计算机工程系） 纠错

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted at IEEE Transactions on Network Science and Engineering