arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.00621cs.CLcs.AI
收藏

基于半监督学习的汉语多音字消歧

Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning

Xmov

Yi Shi, Congyi Wang, Yu Chen, Bin Wang

AI总结:

针对汉语多音字消歧缺乏公开标注数据、多音字规则不规则的难题,提出可利用无标注文本的半监督学习框架,验证多种代理标注策略效果,达到最优性能并发布专用数据集。

摘要:

大多数汉字为单音字,而一类被称为多音字的特殊汉字具有多个读音。作为执行语音相关生成任务的前提,必须从多个候选读音中识别出正确读音,这一过程被称为多音字消歧。尽管基于知识和基于学习的方法已对该问题开展了充分研究,但由于公开可用的标注数据集匮乏,且汉语多音字本身具有不规则性,该问题仍具挑战性。本文提出一种用于汉语多音字消歧的新型半监督学习(SSL)框架,该框架有望利用无限量的无标注文本数据。我们探究了多种代理标注策略的效果,包括熵阈值法和基于词典的标注。定性与定量实验表明,本文方法达到了当前最优性能。此外,我们还发布了一个专门面向多音字消歧任务的新型数据集,以推动后续研究。

英文摘要:

The majority of Chinese characters are monophonic, while a special group of characters, called polyphonic characters, have multiple pronunciations. As a prerequisite of performing speech-related generative tasks, the correct pronunciation must be identified among several candidates. This process is called Polyphone Disambiguation. Although the problem has been well explored with both knowledge-based and learning-based approaches, it remains challenging due to the lack of publicly available labeled datasets and the irregular nature of polyphone in Mandarin Chinese. In this paper, we propose a novel semi-supervised learning (SSL) framework for Mandarin Chinese polyphone disambiguation that can potentially leverage unlimited unlabeled text data. We explore the effect of various proxy labeling strategies including entropy-thresholding and lexicon-based labeling. Qualitative and quantitative experiments demonstrate that our method achieves state-of-the-art performance. In addition, we publish a novel dataset specifically for the polyphone disambiguation task to promote further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑