多模态大模型 - arXivDaily 专题

2504.16102 2025-10-27 cs.CV cs.RO 84%

HAVT-IVD: Heterogeneity-Aware Cross-Modal Network for Audio-Visual Surveillance: Idling Vehicles Detection With Multichannel Audio and Multiscale Visual Cues

Xiwen Li, Xiaoya Tang, Tolga Tasdizen

机构 * Scientific Computing and Imaging Institute（科学计算与成像研究所）； Department of Electrical and Computer Engineering（电气与计算机工程系）； University of Utah（犹他大学）

专题命中音频语音多模态：cross-modal(title)；audio-visual(title)；分类 cs.CV

2510.12995 2025-10-27 eess.AS cs.SD 83%

Continuous-Token Diffusion for Speaker-Referenced TTS in Multimodal LLMs

Xinlu He, Swayambhu Nath Ray, Harish Mallidi, Jia-Hong Huang, Ashwin Bellur, Chander Chandak, M. Maruf, Venkatesh Ravichandran

机构 * Worcester Polytechnic Institute（沃斯特理工大学）； Amazon AGI（亚马逊人工智能实验室）

专题命中音频语音多模态：multimodal(title,abstract)；MLLM(abstract)；分类 eess.AS

2505.11217 2025-10-27 cs.SD cs.AI cs.CV cs.MM eess.AS 77%

Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localization

Yanhao Jia, Ji Xie, S Jivaganesh, Hao Li, Xu Wu, Mengmi Zhang

机构 * Nanyang Technological University, Singapore（南洋理工大学）； Peking University, China（北京大学）； Shenzhen University, China（深圳大学）

专题命中音频语音多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments NeurIPS 2025, Spotlight

2501.01957 2025-10-27 cs.CV cs.SD eess.AS 74%

VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, Long Ma, Xiawu Zheng, Rongrong Ji, Xing Sun, Caifeng Shan, Ran He

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）； School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）； Tencent Youtu Lab（腾讯优图实验室）； XMU（厦门大学）； CASIA（中国科学院自动化研究所）

专题命中音频语音多模态：MLLM(abstract,comments)；multimodal(abstract)；分类 cs.CV、eess.AS

Comments NeurIPS 2025 Spotlight, Code 2.4K Stars: https://github.com/VITA-MLLM/VITA

2505.22088 2025-10-27 cs.SD cs.CL eess.AS 62%

Visual Cues Support Robust Turn-taking Prediction in Noise

Sam O'Connor Russell, Naomi Harte

机构 * ADAPT Centre, School of Engineering（ADAPT中心，工程学院）

专题命中音频语音多模态：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to INTERSPEECH 2025, 10.21437/Interspeech.2025-668

Journal ref Proc. of INTERSPEECH 2025, 1073--1077, 10.21437/Interspeech.2025-668

2510.20850 2025-10-27 eess.AS cs.CL cs.SD 62%

Can large audio language models understand child stuttering speech? speech summarization, and source separation

Chibuzor Okocha, Maya Bakri, Christan Grant

机构 * Department of Computer Science, University of Florida, Gainesville, FL, USA（佛罗里达大学计算机科学系）； Department of Computer Science, Lebanese American University（黎巴嫩美国大学计算机科学系）

专题命中音频语音多模态：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments 7 pages, 1 Figure, 8 tables, Under review ICASSP 2026

2510.21581 2025-10-27 cs.CV cs.SD 57%

Foley Control: Aligning a Frozen Latent Text-to-Audio Model to Video

Ciara Rowles, Varun Jampani, Simon Donné, Shimon Vainer, Julian Parker, Zach Evans

机构 * Stability AI

专题命中音频语音多模态：multi-modal(abstract)；分类 cs.CV

Comments Project Page: https://stability-ai.github.io/foleycontrol.github.io/

2505.21043 2025-10-27 cs.CL cs.RO 57%

Visual Cues Enhance Predictive Turn-Taking for Two-Party Human Interaction

Sam O'Connor Russell, Naomi Harte

机构 * ADAPT Centre, School of Engineering, Trinity College Dublin（ADAPT中心、工程学院、都柏林信任学院）

专题命中音频语音多模态：multimodal(abstract)；分类 cs.CL

Comments Accepted to ACL 2025, Findings of the Association for Computational Linguistics

Journal ref In Findings of the Association for Computational Linguistics: ACL 2025, pages 209--221, Vienna, Austria. Association for Computational Linguistics, 10.18653/v1/2025.findings-acl.12

2410.18972 2025-10-27 cs.LG cs.AI 57%

Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques

David Ortiz-Perez, Manuel Benavent-Lledo, Jose Garcia-Rodriguez, David Tomás, M. Flores Vizcaya-Moreno

机构 * Dept. of Computer Science and Technology（计算机科学与技术系）； University of Alicante（阿尔瓦登特大学）； Unit of Clinical Nursing Research（临床护理研究单位）； Faculty of Health Sciences（健康科学学院）

专题命中音频语音多模态：multimodal(abstract)；分类 cs.AI

Journal ref Applied Soft Computing, Vol. 184, 2025, Article 113787

2510.20867 2025-10-27 cs.LG cs.AI 57%

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards

Jiajun Fan, Roger Ren, Jingyuan Li, Rahul Pandey, Prashanth Gurunath Shivakumar, Ivan Bulyko, Ankur Gandhe, Ge Liu, Yile Gu

机构 * Amazon（亚马逊）； Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（计算机与数据科学学院，伊利诺伊大学厄巴纳-香槟分校）