多模态大模型

Generating Accurate and Detailed Captions for High-Resolution Images

Hankyeol Lee, Gawon Seo, Kyounggyu Lee, Dogun Kim, Kyungwoo Song, Jiyoung Jung

机构 * Department of Artificial Intelligence, University of Seoul（首尔大学人工智能系） ； Department of Computer Science and Engineering, POSTECH（POSTECH计算机科学与工程系） ； Department of Applied Statistics, Yonsei University（延世大学应用统计系） 纠错

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Work conducted in 2024; released for archival purposes

Transformers in Medicine: Improving Vision-Language Alignment for Medical Image Captioning

Yogesh Thakku Suresh, Vishwajeet Shivaji Hogale, Luca-Alexandru Zamfira, Anandavardhana Hegde

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments This work is to appear in the Proceedings of MICAD 2025, the 6th International Conference on Medical Imaging and Computer-Aided Diagnosis

ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models

Xin Tang, Youfang Han, Fangfei Gou, Wei Zhao, Xin Meng, Yang Yu, Jinguo Zhang, Yuanchun Shi, Yuntao Wang, Tengxiang Zhang

机构 * Tsinghua University（清华大学） ； Goertek Inc（歌尔股份有限公司） 纠错

专题命中 图文多模态 ：multimodal(abstract)

Comments 23 pages, 13 figures, 7 tables

Audio-Visual Speech Enhancement In Complex Scenarios With Separation And Dereverberation Joint Modeling

Jiarong Du, Zhan Jin, Peijun Yang, Juan Liu, Zhuo Li, Xin Liu, Ming Li

机构 * School of Cyber Science and Engineering, Wuhan University（武汉大学计算机科学与工程学院） ； School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院） ； School of Computer Science, Wuhan University（武汉大学计算机学院） ； Suzhou Municipal Key Laboratory of Multimodal Intelligent Systems, Digital Innovation Research Center, Duke Kunshan University（多模态智能系统苏州市级重点实验室、杜克昆山大学数字创新研究中心） ； Hardware Engineering System, OPPO（OPPO硬件工程系统） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM、eess.AS

UTI-LLM: A Personalized Articulatory-Speech Therapy Assistance System Based on Multimodal Large Language Model

Yudong Yang, Xiaokang Liu, Shaofeng zhao, Rongfeng Su, Nan Yan, Lan Wang

机构 * Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China（深圳先进技术研究院，中国科学院，中国） ； Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences, China（生物医学成像科学与系统重点实验室，中国科学院，中国） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；MLLM(abstract)

Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy?

Yiwen Guan, Viet Anh Trinh, Vivek Voleti, Jacob Whitehill

专题命中 音频语音多模态 ：multi-modal(title)；分类 cs.CV、cs.CL、cs.MM

Journal ref IEEE International Conference on Multimedia and Expo (ICME), Nantes, France, 2025, pp. 1-6

SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregation

Guangzhi Su, Shuchang Huang, Yutong Ke, Zhuohang Liu, Long Qian, Kaizhu Huang

机构 * Independent Researcher（独立研究者） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)

Inferring trust in recommendation systems from brain, behavioural, and physiological data

Vincent K. M. Cheung, Pei-Cheng Shih, Masato Hirano, Masataka Goto, Shinichi Furuya

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Semantic Frame Aggregation-based Transformer for Live Video Comment Generation

Anam Fatima, Yi Yu, Janak Kapuriya, Julien Lalanne, Jainendra Shukla

机构 * IIIT-Delhi, India（德里印度理工学院） ； Grenoble INP – Ensimag, France（法国格勒诺布尔INP–Ensimag） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Toward Accurate Long-Horizon Robotic Manipulation: Language-to-Action with Foundation Models via Scene Graphs

Sushil Samuel Dinesh, Shinkyu Park

机构 * Department of Electrical and Computer Engineering, King Abdullah University of Science and Technology (KAUST)（电气与计算机工程系，国王 Abdullah 科学与技术大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI

PROFIT: A Specialized Optimizer for Deep Fine Tuning

Anirudh S Chakravarthy, Shuai Kyle Zheng, Xin Huang, Sachithra Hemachandra, Xiao Zhang, Yuning Chai, Zhao Chen

机构 * GM Cruise LLC 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments technical report, 23 pages, NeurIPS 2025 poster

Adaptive Human-Computer Interaction Strategies Through Reinforcement Learning in Complex

Rui Liu, Yifan Zhuang, Runsheng Zhang

专题命中 视频多模态 ：multimodal(abstract)

FAST: Flexible and Adaptive Semantic Transmission for Resource-constrained Multi-user Generative Semantic Communication

Yiru Wang, Wanting Yang, Fangli Mou, Zehui Xiong, Zide Fan, Shiwen Mao, Tony Q. S. Quek

专题命中 视频多模态 ：cross-modal(abstract)

RzenEmbed: Towards Comprehensive Multimodal Retrieval

Weijian Jian, Yajun Zhang, Dawei Liang, Chunyu Xie, Yixiao He, Dawei Leng, Yuhui Yin

机构 * AI Research（360人工智能研究院） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum

Zhuoning Guo, Mingxin Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Xiaowen Chu

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团） 纠错

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV、cs.CL、cs.AI

CogPlanner: Unveiling the Potential of Agentic Multimodal Retrieval Augmented Generation with Planning

Xiaohan Yu, Zhihan Yang, Chong Chen

专题命中 跨模态检索 ：multimodal(title,abstract)

Comments Accepted by SIGIR-AP 2025

Sketch-to-Layout: Sketch-Guided Multimodal Layout Generation

Riccardo Brioschi, Aleksandr Alekseev, Emanuele Nevali, Berkay Döner, Omar El Malki, Blagoj Mitrevski, Leandro Kieliger, Mark Collier, Andrii Maksai, Jesse Berent, Claudiu Musat, Efi Kokiopoulou

机构 * EPFL（苏黎世联邦理工学院） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 18 figures, GitHub link: https://github.com/google-deepmind/sketch_to_layout, accept at ICCV 2025 Workshop (HiGen)

E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources

Tong Shen, Jingai Yu, Dong Zhou, Dong Li, Emad Barsoum

机构 * Advanced Micro Devices, Inc.（Advanced Micro Devices公司） 纠错

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV

GeoFM: Enhancing Geometric Reasoning of MLLMs via Synthetic Data Generation through Formal Language

Yuhao Zhang, Dingxin Hu, Tinghao Yu, Hao Liu, Yiting Liu

机构 * Tencent Hunyuan Team（腾讯文言团队） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

Context-Gated Cross-Modal Perception with Visual Mamba for PET-CT Lung Tumor Segmentation

Elena Mulero Ayllón, Linlin Shen, Pierangelo Veltri, Fabrizia Gelardi, Arturo Chiti, Paolo Soda, Matteo Tortora

机构 * Unit of Artificial Intelligence and Computer Systems, Università Campus Bio-Medico di Roma, Italy（人工智能与计算机系统单位，罗马大学生物医学校园，意大利） ； College of Computer Science and Software Engineering, Shenzhen University, China（计算机科学与软件工程学院，深圳大学，中国） ； Dept. of Computer Engineering, Modeling, Electronic and System Engineering, University of Calabria, Italy（计算机工程、建模、电子与系统工程系，卡拉布里亚大学，意大利） ； IRCCS San Raffaele Hospital, Italy（圣拉斐拉医院，意大利） ； Faculty of Medicine, Vita-Salute San Raffaele University, Italy（医学学院，维塔-桑拉斐拉大学，意大利） ； Dept. of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, Sweden（诊断与干预系，辐射物理，生物医学工程，乌梅大学，瑞典） ； Dept. of Naval, Electrical, Electronics and Telecommunications Engineering, University of Genoa, Italy（海军、电子、电子与电信工程系，热那亚大学，意大利） 纠错

专题命中 多模态评测 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models

Zixin Chen, Hongzhan Lin, Kaixin Li, Ziyang Luo, Yayue Deng, Jing Ma

机构 * Hong Kong Baptist University（香港 Baptist 大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025

Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluation

Galann Pennec, Zhengyuan Liu, Nicholas Asher, Philippe Muller, Nancy F. Chen

机构 * IRIT, University of Toulouse, France（IRIT，图卢兹大学，法国） ； Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I2R），A*STAR，新加坡） ； CNRS, IRIT, France（CNRS，IRIT，法国） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding

Wei Xu, Cheng Wang, Dingkang Liang, Zongchuang Zhao, Xingyu Jiang, Peng Zhang, Xiang Bai

机构 * National University of Defense Technology（国防科技大学） 纠错

专题命中 多模态评测 ：multimodal(title)；image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2025. Data and models are available at https://github.com/H-EmbodVis/NAUTILUS

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

Magdalena Lederbauer, Siddharth Betala, Xiyao Li, Ayush Jain, Amine Sehaba, Georgia Channing, Grégoire Germain, Anamaria Leonescu, Faris Flaifil, Alfonso Amayuelas, Alexandre Nozadze, Stefan P. Schmid, Mohd Zaki, Sudheesh Kumar Ethirajan, Elton Pan, Mathilde Franckel, Alexandre Duval, N. M. Anoop Krishnan, Samuel P. Gleason

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.AI

Comments 29 pages, 13 figures, 6 tables

Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting

ChengAo Shen, Wenchao Yu, Ziming Zhao, Dongjin Song, Wei Cheng, Haifeng Chen, Jingchao Ni

机构 * University of Houston（德克萨斯大学休斯顿分校） ； NEC Laboratories America（日本 NEC 美国实验室） ； University of Connecticut（康涅狄格大学） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.AI

A Multi-Modal Neuro-Symbolic Approach for Spatial Reasoning-Based Visual Grounding in Robotics

Simindokht Jahangard, Mehrzad Mohammadi, Abhinav Dhall, Hamid Rezatofighi

机构 * Faculty of Information Technology, Monash University（信息技术学院，墨尔本大学） ； Department of Electrical Engineering, Sharif University of Technology（电气工程系，谢赫大学） 纠错

专题命中 多模态评测 ：multi-modal(title)；分类 cs.CV、cs.AI

CombiGraph-Vis: A Curated Multimodal Olympiad Benchmark for Discrete Mathematical Reasoning

Hamed Mahdavi, Pouria Mahdavinia, Alireza Farhadi, Pegah Mohammadipour, Samira Malek, Majid Daliri, Pedram Mohammadipour, Alireza Hashemi, Amir Khasahmadi, Vasant Honavar

专题命中 多模态评测 ：multimodal(title)；分类 cs.AI

Comments Code/data: https://github.com/ref-grader/ref-grader, https://huggingface.co/datasets/combviz/inoi

Aeolus: A Multi-structural Flight Delay Dataset

Lin Xu, Xinyun Yuan, Yuxuan Liang, Suwan Yin, Yuankai Wu

机构 * Sichuan University（四川大学） ； The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） 纠错

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.AI

DiagramEval: Evaluating LLM-Generated Diagrams via Graphs

Chumeng Liang, Jiaxuan You

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 多模态评测 ：multimodal(abstract)；分类 cs.CL

Comments EMNLP 2025 Main

$\mathtt{M^3VIR}$: A Large-Scale Multi-Modality Multi-View Synthesized Benchmark Dataset for Image Restoration and Content Creation

Yuanzhi Li, Lebin Zhou, Nam Ling, Zhenghao Chen, Wei Wang, Wei Jiang

机构 * Santa Clara University（圣克拉拉大学） ； University of Newcastle（新castle大学） ； Futurewei Technologies, Inc.（未来科技公司） 纠错

专题命中 多模态评测 ：multi-modal(abstract)；分类 cs.CV