多模态大模型

DA-Mamba: Dialogue-aware selective state-space model for multimodal engagement estimation

Shenwei Kang, Xin Zhang, Wen Liu, Bin Li, Yujie Liu, Bo Gao

机构 * Communication University of China（中国通信大学） ； Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院） ； School of Information Engineering, Beijing Institute of Graphic Communication（北京印刷学院信息工程学院） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation

Jia Li, Yinfeng Yu, Liejun Wang, Fuchun Sun, Wendong Zheng

机构 * Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术创新中心） ； School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院） ； Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； School of Electrical Engineering and Automation, Tianjin University of Technology（天津理工大学电气工程与自动化学院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.AI

Comments Main paper (14 pages). Accepted for publication by ICONIP( International Conference on Neural Information Processing) 2025

The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio

Renhao Wang, Haoran Geng, Tingle Li, Feishi Wang, Gopala Anumanchipalli, Trevor Darrell, Boyi Li, Pieter Abbeel, Jitendra Malik, Alexei A. Efros

机构 * University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Conference on Robot Learning 2025

SwarmChat: An LLM-Based, Context-Aware Multimodal Interaction System for Robotic Swarms

Ettilla Mohiuddin Eumi, Hussein Abbass, Nadine Marcus

机构 * School of Systems & Computing, UNSW Canberra（系统与计算学院，UNSW堪培拉） ； School of Computer Science and Engineering, UNSW Sydney（计算机科学与工程学院，UNSW悉尼） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)

Comments This paper has been accepted and presented at the 16th International Conference on Swarm Intelligence (ICSI 2025), held on July 11-15, 2025, in Yokohama, Japan

Qwen3-Omni Technical Report

Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo Zheng, Rui Men, Fan Zhou, Bowen Yu, Jianxin Yang, Le Yu, Jingren Zhou, Junyang Lin

机构 * Qwen Team（通义实验室） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments https://github.com/QwenLM/Qwen3-Omni

Leveraging Audio-Visual Data to Reduce the Multilingual Gap in Self-Supervised Speech Models

María Andrea Cruz Blandón, Zakaria Aldeneh, Jie Chi, Maureen de Seyssel

机构 * Tampere University Apple（塔尔库大学苹果） 纠错

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CL、eess.AS

Comments 5 pages, 2 figures

PGSTalker: Real-Time Audio-Driven Talking Head Generation via 3D Gaussian Splatting with Pixel-Aware Density Control

Tianheng Zhu, Yinfeng Yu, Liejun Wang, Fuchun Sun, Wendong Zheng

机构 * Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术创新研究中心） ； School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院） ； Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； School of Electrical Engineering and Automation, Tianjin University of Technology（天津理工大学电气工程与自动化学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.AI

Comments Main paper (15 pages). Accepted for publication by ICONIP( International Conference on Neural Information Processing) 2025

Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models

Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, Chunyan Miao

机构 * Nanyang Technological University（南洋理工大学） ； National University of Singapore（国立新加坡大学） ； Rakuten 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI、cs.MM

Comments Technical report, in process

Whisper-UT: A Unified Translation Framework for Speech and Text

Cihan Xiao, Matthew Wiesner, Debashish Chakraborty, Reno Kriz, Keith Cunningham, Kenton Murray, Kevin Duh, Luis Tavarez-Arce, Paul McNamee, Sanjeev Khudanpur

机构 * Center for Language and Speech Processing, Johns Hopkins University（语言与语音处理中心，约翰霍普金斯大学） ； Human Language Technology Center of Excellence, Johns Hopkins University（人类语言技术卓越中心，约翰霍普金斯大学） ； Georgetown University（乔治城大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL

Comments EMNLP 2025 Main Conference

VAInpaint: Zero-Shot Video-Audio inpainting framework with LLMs-driven Module

Kam Man Wu, Zeyue Tian, Liya Ji, Qifeng Chen

机构 * The Hong Kong University of Science and Technology（香港科学与技术大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM、eess.AS

Speech-to-See: End-to-End Speech-Driven Open-Set Object Detection

Wenhuan Lu, Xinyue Song, Wenjun Ke, Zhizhi Yu, Wenhao Yang, Jianguo Wei

机构 * College of Intelligence and Computing, Tianjin University, Tianjin, China（智能与计算学院，天津大学，天津，中国） ； PipeChina Institute of Science and Technology, Tianjin, China（中石油科技研究院，天津，中国） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

STAR: Speech-to-Audio Generation via Representation Learning

Zeyu Xie, Xuenan Xu, Yixuan Li, Mengyue Wu, Yuexian Zou

机构 * Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Peking University, Shenzhen（广东超高清沉浸媒体技术重点实验室，北京大学深圳校区） ； X-LANCE Lab, Shanghai Jiao Tong University, Shanghai（X-LANCE实验室，上海交通大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

What's Not on the Plate? Rethinking Food Computing through Indigenous Indian Datasets

Pamir Gogoi, Neha Joshi, Ayushi Pandey, Deepthi Sudharsan, Saransh Kumar Gupta, Lipika Dey, Partha Pratim Das, Kalika Bali, Vivek Seshadri

专题命中 音频语音多模态 ：multimodal(abstract)