Code-switching (CS) refers to the phenomenon that languages switch within a speech signal and leads to language confusion for automatic speech recognition (ASR). This paper aims to address language confusion for improving CS-ASR from two perspectives: incorporating and disentangling language information. We incorporate language information in the CS-ASR model by dynamically biasing the model with token-level language posteriors which are outputs of a sequence-to-sequence auxiliary language diarization module. In contrast, the disentangling process reduces the difference between languages via adversarial training so as to normalize two languages. We conduct the experiments on the SEAME dataset. Compared to the baseline model, both the joint optimization with LD and the language posterior bias achieve performance improvement. The comparison of the proposed methods indicates that incorporating language information is more effective than disentangling for reducing language confusion in CS speech.
Reducing language context confusion for end-to-end code-switching automatic speech recognition
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；Huawei Noah's Ark Lab（华为诺亚方舟实验室）
语码转换是指在交际过程中出现的替代语言。训练用于代码切换的端到端（E2 E）自动语音识别（ASR）系统尤其具有挑战性，因为代码切换训练数据总是不足以对抗由于存在多于一种语言而增加的多语言上下文混淆。本文基于等价约束理论，提出了一种与语言相关的注意力机制，以减少E2 E语码转换ASR模型中的多语言语境混淆。语言学理论要求任何出现在语码转换句中的单语片段必须出现在单语句子中。该理论在单语数据和语码转换数据之间架起了一座桥梁。我们利用这一语言学理论来设计语码转换E2 E ASR模型。该模型有效地从丰富的单语数据中转移语言知识，以提高代码转换ASR模型的性能。我们在ASRU 2019普通话-英语码转换挑战数据集上评估了我们的模型。与基线模型相比，我们提出的模型实现了17.12%的相对误差减少。
Speaker Diarization for Low-Resource Languages Through Wav2vec Fine-Tuning
Artificial Intelligence and Innovation Centre, University of Kurdistan Hewler（库尔德斯坦 Hewler 大学人工智能与创新中心）；College of Science, University of Halabja（哈拉布贾大学理学院）；Komar University of Science and Technology（科马尔科技大学）；Technical College of Informatics, Sulaimani Polytechnic University（苏莱曼尼理工大学信息技术学院）
