arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.13655cs.CL
收藏

中文拼写检查的综合评估与分析研究

A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check

Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Center for Data Science, Peking University(北京大学数据科学中心);The MOE Key Laboratory of Computational Linguistics, Peking University(北京大学计算语言学教育部重点实验室)

Xunjian Yin, Xiaojun Wan

AI总结:

本研究通过九种结构实现代表性模型范式,在综合测试集上评估中文拼写检查模型,发现融合语音和图形信息有效,模型对错误分布敏感,且SIGHAN基准不可靠。

摘要:

随着预训练模型的发展以及语音和图形信息的融入,神经模型在中文拼写检查(CSC)中取得了高分。然而,由于测试集有限,这并不能全面反映模型的能力。在本研究中,我们抽象了代表性模型范式,用九种结构实现它,并在我们构建的具有不同目的的综合测试集上进行了实验。我们对结果进行了详细分析,发现:1)合理融合语音和图形信息对CSC是有效的。2)模型对测试集的错误分布敏感,这反映了模型的缺点,并揭示了我们应该努力的方向。3)错误和上下文是否被见过对模型有显著影响。4)常用的基准SIGHAN不能可靠地评估模型的性能。

英文摘要:

With the development of pre-trained models and the incorporation of phonetic and graphic information, neural models have achieved high scores in Chinese Spelling Check (CSC). However, it does not provide a comprehensive reflection of the models' capability due to the limited test sets. In this study, we abstract the representative model paradigm, implement it with nine structures and experiment them on comprehensive test sets we constructed with different purposes. We perform a detailed analysis of the results and find that: 1) Fusing phonetic and graphic information reasonably is effective for CSC. 2) Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models and reveals the direction we should work on. 3) Whether or not the errors and contexts have been seen has a significant impact on models. 4) The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑