XANE:可解释的声学神经嵌入
XANE: eXplainable Acoustic Neural Embeddings
AI总结:
本文提出XANE方法,通过提取可解释的神经嵌入建模语音背景声学,在聚类任务中平均F1达95.2%,优于WavLM,并能以低17倍实时因子高精度估计14个背景声学参数。
摘要:
我们提出了一种提取神经嵌入的新方法,用于对语音信号的背景声学进行建模。所提取的嵌入以非侵入方式用于估计与信号背景声学特性相关的特定参数,从而使嵌入能够依据这些参数进行解释。我们通过在未见测试数据上进行聚类实验来说明这些嵌入的价值,并表明所提出的嵌入在三个不同任务上实现了95.2%的平均F1分数,显著优于基于WavLM的信号嵌入。我们还表明,所提出的方法能够通过估计14个表征背景声学的声学参数来解释嵌入,包括混响和噪声水平、重叠语音检测、CODEC类型检测和噪声类型检测,且精度高,实时因子比外部基线方法低17倍。
英文摘要:
We present a novel method for extracting neural embeddings that model the background acoustics of a speech signal. The extracted embeddings are used to estimate specific parameters related to the background acoustic properties of the signal in a non-intrusive manner, which allows the embeddings to be explainable in terms of those parameters. We illustrate the value of these embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 95.2\% for three different tasks, outperforming significantly the WavLM based signal embeddings. We also show that the proposed method can explain the embeddings by estimating 14 acoustic parameters characterizing the background acoustics, including reverberation and noise levels, overlapped speech detection, CODEC type detection and noise type detection with high accuracy and a real-time factor 17 times lower than an external baseline method.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。