arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.15732cs.CLcs.SDeess.AS
收藏

假设聚类与合并:基于说话人Token的全新多人语音识别

Hypothesis Clustering and Merging: Novel MultiTalker Speech Recognition with Speaker Tokens

Sony Group Corporation(索尼集团公司);Carnegie Mellon University(卡内基梅隆大学)

Yosuke Kashiwagi, Hayato Futami, Emiru Tsunoo, Siddhant Arora, Shinji Watanabe

AI总结:

针对未知人数且语音重叠的多说话人识别问题,提出基于说话人聚类token的编码器-解码器方法,通过AHC合并假设,在LibriMix 3-mix环境中相比传统方法显著降低错误率。

摘要:

在会议等许多真实场景中,通常存在未知数量的说话人,且他们的语音常常重叠。我们通过一种新颖的基于注意力的编码器-解码器方法来解决这些多说话人挑战,该方法通过说话人聚类获得了特殊的说话人类别token。在推理过程中,我们选择基于预测的说话人聚类token的多个识别假设,并基于归一化编辑距离通过凝聚层次聚类(AHC)合并这些假设。聚类后的假设产生多说话人转录,说话人数量由AHC确定。我们在LibriMix数据集上的实验表明,我们提出的方法在复杂的3-mix环境中特别有效,与传统的串行输出训练相比,在干净数据上实现了55%的相对错误减少,在嘈杂数据上实现了36%的相对错误减少。

英文摘要:

In many real-world scenarios, such as meetings, multiple speakers are present with an unknown number of participants, and their utterances often overlap. We address these multi-speaker challenges by a novel attention-based encoder-decoder method augmented with special speaker class tokens obtained by speaker clustering. During inference, we select multiple recognition hypotheses conditioned on predicted speaker cluster tokens, and these hypotheses are merged by agglomerative hierarchical clustering (AHC) based on the normalized edit distance. The clustered hypotheses result in the multi-speaker transcriptions with the appropriate number of speakers determined by AHC. Our experiments on the LibriMix dataset demonstrate that our proposed method was particularly effective in complex 3-mix environments, achieving a 55% relative error reduction on clean data and a 36% relative error reduction on noisy data compared with conventional serialized output training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑