IPA 的味道：迈向任意语言中的开放词汇关键词识别与强制对齐
The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any language
AI总结：
本文提出基于音素-语音对比嵌入的 CLAP-IPA 模型及 IPA-ALIGNER 强制对齐器，并构建覆盖 115 种以上语言的 IPAPACK 语料库，在 95 种未见语言上实现开放词汇关键词识别与零样本强制对齐。
摘要：
在本项目中，我们证明了基于音素的语音处理模型能够对未见过的语言表现出很强的跨语言泛化能力。我们构建了 IPAPACK，一个大规模多语言语音语料库，包含音素转写，涵盖来自不同语系的 115 种以上语言，并经过语言学家的选择性检查。基于 IPAPACK，我们提出了 CLAP-IPA，一种多语言音素-语音对比嵌入模型，能够在任意语音信号与音素序列之间进行开放词汇匹配。该模型在 95 种未见过的语言上进行了测试，展现出很强的跨语言泛化能力。音素与语音信号之间的时间对齐也从对比训练中自然涌现，从而能够在未见过的语言中实现零样本强制对齐。我们进一步引入了一个神经强制对齐器 IPA-ALIGNER，通过使用 Forward-Sum 损失对 CLAP-IPA 进行微调，以学习更好的音素到音频的对齐。评估结果表明，IPA-ALIGNER 无需适配即可泛化到未见过的语言。
英文摘要：
In this project, we demonstrate that phoneme-based models for speech processing can achieve strong crosslinguistic generalizability to unseen languages. We curated the IPAPACK, a massively multilingual speech corpora with phonemic transcriptions, encompassing more than 115 languages from diverse language families, selectively checked by linguists. Based on the IPAPACK, we propose CLAP-IPA, a multi-lingual phoneme-speech contrastive embedding model capable of open-vocabulary matching between arbitrary speech signals and phonemic sequences. The proposed model was tested on 95 unseen languages, showing strong generalizability across languages. Temporal alignments between phonemes and speech signals also emerged from contrastive training, enabling zeroshot forced alignment in unseen languages. We further introduced a neural forced aligner IPA-ALIGNER by finetuning CLAP-IPA with the Forward-Sum loss to learn better phone-to-audio alignment. Evaluation results suggest that IPA-ALIGNER can generalize to unseen languages without adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
