This paper introduces a phase-aware probabilistic model for audio source separation. Classical source models in the short-term Fourier transform domain use circularly-symmetric Gaussian or Poisson random variables. This is equivalent to assuming that the phase of each source is uniformly distributed, which is not suitable for exploiting the underlying structure of the phase. Drawing on preliminary works, we introduce here a Bayesian anisotropic Gaussian source model in which the phase is no longer uniform. Such a model permits us to favor a phase value that originates from a signal model through a Markov chain prior structure. The variance of the latent variables are structured with nonnegative matrix factorization (NMF). The resulting model is called complex Itakura-Saito NMF (ISNMF) since it generalizes the ISNMF model to the case of non-isotropic variables. It combines the advantages of ISNMF, which uses a distortion measure adapted to audio and yields a set of estimates which preserve the overall energy of the mixture, and of complex NMF, which enables one to account for some phase constraints. We derive a generalized expectation-maximization algorithm to estimate the model parameters. Experiments conducted on a musical source separation task in a semi-informed setting show that the proposed approach outperforms state-of-the-art phase-aware separation techniques.
A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio
Microsoft Corp.(微软公司)
说话人属性自动语音识别(SA-ASR)是从多个说话人的录音中识别“谁说了什么”的任务。SA-ASR系统通常由语音分离、说话人日志化和ASR等多个模块组成。另一方面,考虑到联合优化,最近提出了一种端到端(E2 E)SA-ASR模型,并在仿真数据上取得了令人满意的结果。在本文中,我们提出了我们最近的研究比较真实的单声道录音,这种模块化和联合的方法对SA-ASR。我们通过利用大规模训练数据,包括7.5万小时的ASR训练数据和用于说话人表示学习的VoxCeleb语料库,为模块化和联合方法开发最先进的SA-ASR系统。我们还提出了一个新的管道,执行E2 E SA-ASR模型后,扬声器聚类。在AMI会议语料库上的评价表明,经过少量的实际数据微调后,联合系统的准确率比最好的模块系统提高了8.9- 29.9%,而模块系统在微调前的准确率更高。我们还进行了各种误差分析,以显示单耳SA-ASR的剩余问题。
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Department of Computer Science, The University of Texas at Dallas(德克萨斯大学达拉斯分校计算机科学系)
Oberlin College and Conservatory(欧柏林学院与音乐学院);Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);New Mexico Institute of Mining and Technology(新墨西哥矿业技术学院);New Mexico Consortium(新墨西哥联合体)
School of Marine Science and Technology, Northwestern Polytechnical University(西北工业大学海洋科学与技术学院);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院(TeleAI));Research and Development Institute of Northwestern Polytechnical University in Shenzhen(西北工业大学深圳研究院)
Spectro-Temporal Interference Confounds Phase Encoding in Spatial Audio Foundation Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Jilin University(吉林大学);Hunan University(湖南大学);University of Electronic Science and Technology of China(电子科技大学)
School of Marine Science and Technology, Northwestern Polytechnical University(海洋科学与技术学院,西北工业大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);Research and Development Institute of Northwestern Polytechnical University in Shenzhen, China(西北工业大学深圳研发院,中国)
IsoNet: Face-Guided Speech Extraction for Compact Microphone Arrays
Department of Electronics and Computer Engineering, Thapathali Campus, Institute of Engineering, Tribhuvan University(电子与计算机工程系,Thapathali校区,工程学院,塔波胡万大学)