多模态大模型

Deep Learning for Tuberculosis Screening in a High-burden Setting using Cough Analysis and Speech Foundation Models

Ning Ma, Bahman Mirheidari, Guy J. Brown, Nsala Sanjase, Minyoi M. Maimbolwa, Solomon Chifwamba, Seke Muzazu, Monde Muyoyeta, Mary Kagujje

机构 * School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院） ； Centre for Infectious Disease Research in Zambia（赞比亚传染病疾病研究中心） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments submitted to IEEE Journal of Biomedical and Health Informatics

Video Object Segmentation-Aware Audio Generation

Ilpo Viertola, Vladimir Iashin, Esa Rahtu

机构 * Tampere University, Tampere, Finland（塔尔库大学） ； University of Oxford, Oxford, UK（牛津大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Preprint version. The Version of Record is published in DAGM GCPR 2025 proceedings with Springer Lecture Notes in Computer Science (LNCS). Updated results and resources are available at the project page: https://saganet.notion.site

LTA-L2S: Lexical Tone-Aware Lip-to-Speech Synthesis for Mandarin with Cross-Lingual Transfer Learning

Kang Yang, Yifan Liang, Fangkun Liu, Zhenping Xie, Chengshi Zheng

机构 * School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院） ； Institute of Acoustics Chinese Academy of Science（中国科学院声学研究所） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Submitted to ICASSP 2026

Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling

Neil Zeghidour, Eugene Kharitonov, Manu Orsini, Václav Volhejn, Gabriel de Marmiesse, Edouard Grave, Patrick Pérez, Laurent Mazaré, Alexandre Défossez

机构 * Kyutai 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling

Chao Huang, Susan Liang, Yapeng Tian, Anurag Kumar, Chenliang Xu

机构 * Meta Reality Labs Research（Meta现实实验室） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Accepted to IJCV

Comprehend and Talk: Text to Speech Synthesis via Dual Language Modeling

Junjie Cao, Yichen Han, Ruonan Zhang, Xiaoyang Hao, Hongxiang Li, Shuaijiang Zhao, Yue Liu, Xiao-Ping Zhng

机构 * Tsinghua University（清华大学） ； Peking University（北京大学） ； AMAP Speech（AMAP语音） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments conference paper about TTS

X-Streamer: Unified Human World Modeling with Audiovisual Interaction

You Xie, Tianpei Gu, Zenan Li, Chenxu Zhang, Guoxian Song, Xiaochen Zhao, Chao Liang, Jianwen Jiang, Hongyi Xu, Linjie Luo

机构 * ByteDance（字节跳动） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Project Page at https://byteaigc.github.io/X-Streamer

GLip: A Global-Local Integrated Progressive Framework for Robust Visual Speech Recognition

Tianyue Wang, Shuang Yang, Shiguang Shan, Xilin Chen

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院人工智能安全国家重点实验室，计算技术研究所） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning

Zhilin Wang, Zhe Yang, Yun Luo, Yafu Li, Xiaoye Qu, Ziqian Qiao, Haoran Zhang, Runzhe Zhan, Derek F. Wong, Jizhe Zhou, Yu Cheng

机构 * University of Science and Technology of China（中国科学技术大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； Sichuan University（四川大学） ； Shanghai Jiao Tong University（上海交通大学） ； University of Macau（澳门大学） ； Tsinghua University（清华大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments 34 pages

"She was useful, but a bit too optimistic": Augmenting Design with Interactive Virtual Personas

Paluck Deep, Monica Bharadhidasan, A. Baki Kocaballi

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments The version accepted for publication at International Journal of Human-Computer Studies

Journal ref International Journal of Human-Computer Studies (2025)

On Class Separability Pitfalls In Audio-Text Contrastive Zero-Shot Learning

Tiago Tavares, Fabio Ayres, Zhepei Wang, Paris Smaragdis

机构 * INSPER Institute of Teaching and Research（INSPER教学与研究机构） ； Research University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校研究大学） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice

Sitong Cheng, Weizhen Bian, Xinsheng Wang, Ruibin Yuan, Jianyi Chen, Shunshun Yin, Yike Guo, Wei Xue

机构 * Hong Kong University of Science and Technology（香港理工大学） ； Soul AI Lab（Soul AI 实验室） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.AI

Lessons Learnt: Revisit Key Training Strategies for Effective Speech Emotion Recognition in the Wild

Jing-Tong Tzeng, Bo-Hao Su, Ya-Tse Wu, Hsing-Hang Chou, Chi-Chun Lee

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Proceedings of Interspeech 2025

STAR: Speech-to-Audio Generation via Representation Learning

Zeyu Xie, Xuenan Xu, Yixuan Li, Mengyue Wu, Yuexian Zou

机构 * Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Peking University, Shenzhen（广东超高清沉浸媒体技术重点实验室，北京大学深圳校区） ； X-LANCE Lab, Shanghai Jiao Tong University, Shanghai（X-LANCE实验室，上海交通大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Audio-Based Crowd-Sourced Evaluation of Machine Translation Quality

Sami Ul Haq, Sheila Castilho, Yvette Graham

机构 * ADAPT Centre（ADAPT中心） ； Dublin City University (DCU)（都柏林城市大学） ； Trinity College Dublin (TCD)（三一学院都柏林） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Accepted at WMT2025 (ENNLP) for oral presented

Standards in the Preparation of Biomedical Research Metadata: A Bridge2AI Perspective

Harry Caufield, Satrajit Ghosh, Sek Wong Kong, Jillian Parker, Nathan Sheffield, Bhavesh Patel, Andrew Williams, Timothy Clark, Monica C. Munoz-Torres

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

TextlessRAG: End-to-End Visual Document RAG by Speech Without Text

Peijin Xie, Shun Qian, Bingquan Liu, Dexin Wang, Lin Sun, Xiangzheng Zhang

机构 * IEEE 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments 5 pages, 4 figures,

I2TTS: Image-indicated Immersive Text-to-speech Synthesis with Spatial Perception

Jiawei Zhang, Tian-Hao Zhang, Jun Wang, Jiaran Gao, Xinyuan Qian, Xu-Cheng Yin

机构 * University of Science and Technology Beijing（北京科技大学） ； Tencent AI Lab（腾讯AI实验室） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Accepted by APSIPA ASC2025

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land

Simone Scardapane

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments Companion website for additional chapters: https://www.sscardapane.it/alice-book

Speech Emotion Recognition via Entropy-Aware Score Selection

ChenYi Chua, JunKai Wong, Chengxin Chen, Xiaoxiao Miao

机构 * Singapore Institute of Technology（新加坡理工学院） ； Institute Of Acoustics, Chinese Academy Of Sciences（中国科学院声学研究所） ； Duke Kunshan University（杜克大学昆山分校） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments The paper has been accepted by APCIPA ASC 2025

A Solution of Ultra Wideband Based High-resolution and Lossless Audio Transmission

Fengyun Zhang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments 5 pages

Modality-Specific Speech Enhancement and Noise-Adaptive Fusion for Acoustic and Body-Conduction Microphone Framework

Yunsik Kim, Yoonyoung Chung

机构 * Department of Electrical Engineering（电气工程系） ； Department of Semiconductor Engineering（半导体工程系） ； Center for Semiconductor Technology Convergence（半导体技术融合中心） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Journal ref Interspeech 2025

Large Language Models for Depression Recognition in Spoken Language Integrating Psychological Knowledge

Yupei Li, Shuaijie Shao, Manuel Milling, Björn W. Schuller

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Journal ref Frontiers in Computer Science, Volume 7, 2025

Visuo-Acoustic Hand Pose and Contact Estimation

Yuemin Mao, Uksang Yoo, Yunchao Yao, Shahram Najam Syed, Luca Bondi, Jonathan Francis, Jean Oh, Jeffrey Ichnowski

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence

Sonal Kumar, Šimon Sedláček, Vaibhavi Lokegaonkar, Fernando López, Wenyi Yu, Nishit Anand, Hyeonggon Ryu, Lichang Chen, Maxim Plička, Miroslav Hlaváček, William Fineas Ellingwood, Sathvik Udupa, Siyuan Hou, Allison Ferner, Sara Barahona, Cecilia Bolaños, Satish Rahi, Laura Herrera-Alarcón, Satvik Dixit, Siddhi Patil, Soham Deshmukh, Lasha Koroshinadze, Yao Liu, Leibny Paola Garcia Perera, Eleni Zanou, Themos Stafylakis, Joon Son Chung, David Harwath, Chao Zhang, Dinesh Manocha, Alicia Lozano-Diez, Santosh Kesiraju, Sreyan Ghosh, Ramani Duraiswami

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Can Large Language Models (LLMs) Describe Pictures Like Children? A Comparative Corpus Study

Hanna Woloszyn, Benjamin Gagl

机构 * Self-Learning Systems Lab, Faculty of Human Sciences, Department of Special Education and Rehabilitation（自主学习系统实验室，人文科学学院，特殊教育与康复系） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Cryfish: On deep audio analysis with Large Language Models

Anton Mitrofanov, Sergei Novoselov, Tatiana Prisyach, Vladislav Marchevskiy, Arseniy Karelin, Nikita Khmelev, Dmitry Dutov, Stepan Malykh, Igor Agafonov, Aleksandr Nikitin, Oleg Petrov

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Journal ref Proc. Interspeech 2025

FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation

MengChao Wang, Qiang Wang, Fan Jiang, Mu Xu

机构 * Project Leader（项目负责人） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments https://fantasy-amap.github.io/fantasy-talking2/

Advances in Speech Separation: Techniques, Challenges, and Future Trends

Kai Li, Guo Chen, Wendi Sang, Yi Luo, Zhuo Chen, Shuai Wang, Shulin He, Zhong-Qiu Wang, Andong Li, Zhiyong Wu, Xiaolin Hu

机构 * Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学） ； School of Computer Technology and Application, Qinghai University（计算机技术与应用学院，青海大学） ； ByteDance（字节跳动） ； Nanjing University（南京大学） ； Southern University of Science and Technology（南方科技大学） ； Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments 34 pages, 10 figures

Ensembling Synchronisation-based and Face-Voice Association Paradigms for Robust Active Speaker Detection in Egocentric Recordings

Jason Clarke, Yoshihiko Gotoh, Stefan Goetze

机构 * Speech and Hearing (SPandH), School of Computer Science, The University of Sheffield, UK（语音与听力（SPandH）、计算机科学学院、谢菲尔德大学、英国） ； South Westphalia University of Applied Sciences, Iserlohn, Germany（西南弗兰肯应用科学大学、伊塞尔洛恩、德国） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM

Comments Accepted to SPECOM 2025, 13 pages, 4 figures. To appear in the Proceedings of the 27th International Conference on Speech and Computer (SPECOM) 2025, October 13-14, 2025, Szeged, Hungary