arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

基于自监督语音表示的音素级与词级指标用于强制对齐评估

Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation

V. S. D. S. Mahesh Akavarapu, Michael Daniel, Gerhard Jäger

arXiv 2608.28508首次发表:更新:

发表机构

University of Tübingen; University of Jena(蒂宾根大学; 耶拿大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究提出基于自监督语音表示的PCMI和WACS指标,用于无需人工标注时间戳的可扩展强制对齐评估,可区分对齐质量并与基准高度相关,已开源。

AI 中文摘要

强制对齐评估通常需要人工标注的时间戳,这限制了大规模和多语言分析。我们引入了两个基于自监督(SSL)语音表示的语料库级指标,用于无参考强制对齐评估:音素聚类互信息(PCMI)和词声学一致性分数(WACS)。PCMI测量对齐的音素标签与从SSL语音表示中诱导出的聚类之间的一致性,而WACS使用词表示序列之间的动态时间规整相似度来测量重复词实现的一致性。通过随机和系统扰动,我们表明PCMI和WACS在对齐扰动下会持续下降。我们进一步在来自FLEURS的85种语言上分析这些指标,针对DoReCo中45种语言的人工标注对齐进行验证,并在两种音系复杂的低资源语言上进行评估。这些指标能有效区分高质量和低质量对齐,且与基于时间戳的对齐质量测量高度相关。我们的结果表明,SSL语音表示可实现可扩展的无参考强制对齐评估,这些指标作为开源Python包提供,链接在此。

英文摘要

Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at https://github.com/mahesh-ak/forced-aligner-metrics.

CommentsAccepted at EMNLP-2026 (Findings)

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑