利用挎斗分离器统一建模多说话人重叠语音识别与说话人分离
Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
AI总结：
本文提出在冻结的ASR模型中插入带说话人分离分支的挎斗分离器，以仅768个参数的极小开销统一建模重叠语音识别与说话人分离，并在多个数据集上验证了有效性。
摘要：
多说话人重叠语音对语音识别和说话人分离构成了重大挑战。近期研究表明，这两项任务相互依赖且互补，这促使我们探索一种统一建模方法，在重叠语音场景下同时解决这两个问题。一项近期研究提出了一种经济高效的方法，通过将挎斗（Sidecar）分离器插入到冻结的、训练良好的单说话人自动语音识别（ASR）模型中，将其转换为多说话人系统。在此基础上，我们将一个说话人分离分支整合到挎斗中，从而以仅768个参数的极小开销实现ASR和说话人分离的统一建模。所提出的方法在LibriMix和LibriSpeechMix数据集上相比基线取得了更好的ASR结果。此外，在未对说话人分离任务进行复杂定制的情况下，我们的方法仅需少量自适应步骤即可在CALLHOME的双说话人子集上取得可接受的说话人分离结果。
英文摘要：
Multi-talker overlapped speech poses a significant challenge for speech recognition and diarization. Recent research indicated that these two tasks are inter-dependent and complementary, motivating us to explore a unified modeling method to address them in the context of overlapped speech. A recent study proposed a cost-effective method to convert a single-talker automatic speech recognition (ASR) system into a multi-talker one, by inserting a Sidecar separator into the frozen well-trained ASR model. Extending on this, we incorporate a diarization branch into the Sidecar, allowing for unified modeling of both ASR and diarization with a negligible overhead of only 768 parameters. The proposed method yields better ASR results compared to the baseline on LibriMix and LibriSpeechMix datasets. Moreover, without sophisticated customization on the diarization task, our method achieves acceptable diarization results on the two-speaker subset of CALLHOME with only a few adaptation steps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
