面向语音与说话人识别的多任务学习研究
Towards multi-task learning of speech and speaker recognition
AI总结：
本文研究基于 wav2vec2 的语音与说话人识别多任务学习，发现共享嵌入虽在分布内表现与单任务模型相当，但在分布外数据上性能显著下降。
摘要：
我们研究了两个正交语音技术任务的多任务学习：语音识别和说话人识别。我们使用 wav2vec2 作为基础架构，并配备两个任务特定的输出头。我们尝试了不同的架构决策，以在输出序列中混合说话人和语音信息，并尝试了不同的优化策略。我们的多任务学习网络能够生成共享的说话人和语音嵌入，初步来看其性能与单独的单任务模型相当。然而，我们表明，与单任务模型相比，多任务网络在分布外评估数据上的性能严重下降。代码和模型检查点可在 https://github.com/nikvaessen/disjoint-mtl 获取。
英文摘要：
We study multi-task learning for two orthogonal speech technology tasks: speech and speaker recognition. We use wav2vec2 as a base architecture with two task-specific output heads. We experiment with different architectural decisions to mix speaker and speech information in the output sequence as well as different optimization strategies. Our multi-task learning networks can produce a shared speaker and speech embedding, which on first glance achieve a performance comparable to separate single-task models. However, we show that the multi-task networks have strongly degraded performance on out-of-distribution evaluation data compared to the single-task models. Code and model checkpoints are available at https://github.com/nikvaessen/disjoint-mtl
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
