arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.02859eess.AScs.SD
收藏

ConPCO:利用对比序数正则化保留音素特征以实现自动发音评估

ConPCO: Preserving Phoneme Characteristics for Automatic Pronunciation Assessment Leveraging Contrastive Ordinal Regularization

Bi-Cheng Yan, Wei-Cheng Chao, Jiun-Ting Li, Yi-Cheng Wang, Hsin-Wei Wang, Meng-Shin Lin, Berlin Chen

AI总结:

本文提出对比音素序数正则化器ConPCO,通过对比学习对齐音素表示与文本嵌入,并在特征空间中保留音素类别距离和序数关系,配合分层APA模型在speechocean762上验证了其有效性。

摘要:

自动发音评估(APA)旨在评估第二语言(L2)学习者在目标语言中的发音熟练程度。现有工作通常借助回归模型进行熟练度分数预测,这些模型在训练时仅估计目标值,而未在特征空间中显式考虑音素感知。本文提出了一种专为基于回归的APA模型设计的对比音素序数正则化器(ConPCO),旨在生成更具音素区分性的特征,同时考虑回归目标之间的序数关系。所提出的ConPCO首先通过对比学习对齐APA模型的音素表示与音标转写的文本嵌入。随后,通过在特征空间中调节音素类别间和类别内的距离来保留音素特征,同时兼顾输出目标之间的序数关系。我们进一步设计并开发了一个分层APA模型,以评估所提方法的有效性。在speechocean762基准数据集上进行的大量实验表明,与一些前沿基线方法相比,我们的方法具有可行性和有效性。

英文摘要:

Automatic pronunciation assessment (APA) manages to evaluate the pronunciation proficiency of a second language (L2) learner in a target language. Existing efforts typically draw on regression models for proficiency score prediction, where the models are trained to estimate target values without explicitly accounting for phoneme-awareness in the feature space. In this paper, we propose a contrastive phonemic ordinal regularizer (ConPCO) tailored for regression-based APA models to generate more phoneme-discriminative features while considering the ordinal relationships among the regression targets. The proposed ConPCO first aligns the phoneme representations of an APA model and textual embeddings of phonetic transcriptions via contrastive learning. Afterward, the phoneme characteristics are retained by regulating the distances between inter- and intra-phoneme categories in the feature space while allowing for the ordinal relationships among the output targets. We further design and develop a hierarchical APA model to evaluate the effectiveness of our method. Extensive experiments conducted on the speechocean762 benchmark dataset suggest the feasibility and efficacy of our approach in relation to some cutting-edge baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑