arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.09007cs.CL
收藏

论大型语言模型在中文文本纠错中的(不)有效性

On the (In)Effectiveness of Large Language Models for Chinese Text Correction

Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);OPPO Research Institute(OPPO研究院)

Yinghui Li, Haojing Huang, Shirong Ma, Yong Jiang, Yangning Li, Feng Zhou, Hai-Tao Zheng, Qingyu Zhou

AI总结:

本文系统评估并微调多种代表性 LLM,在 CGEC 与 CSC 任务上对比先进小模型,揭示其在中文文本纠错中的优势与不足。

摘要:

近年来,大型语言模型(LLM)的发展与进步震惊了整个人工智能领域。凭借其涌现能力,LLM 吸引了越来越多研究者考察其在各类下游自然语言处理(NLP)任务上的能力与表现。在惊叹于 LLM 在各种任务上惊人表现的同时,我们注意到它们还具备出色的多语言处理能力,例如中文。为探索 LLM 的中文处理能力,我们聚焦于中文文本纠错这一基础且富有挑战性的中文 NLP 任务。具体而言,我们在中文语法错误纠正(CGEC)和中文拼写检查(CSC)这两个主要的中文文本纠错场景上评估了多种具有代表性的 LLM。此外,我们还针对中文文本纠错对 LLM 进行了微调,以便更好地观察其潜在能力。通过大量分析以及与以往最先进的小模型进行比较,我们从实证角度发现,当前 LLM 在中文文本纠错上既表现出令人惊叹的性能,也存在不尽如人意的行为。我们相信这些发现将推动 LLM 在中文 NLP 社区的落地与应用。

英文摘要:

Recently, the development and progress of Large Language Models (LLMs) have amazed the entire Artificial Intelligence community. Benefiting from their emergent abilities, LLMs have attracted more and more researchers to study their capabilities and performance on various downstream Natural Language Processing (NLP) tasks. While marveling at LLMs' incredible performance on all kinds of tasks, we notice that they also have excellent multilingual processing capabilities, such as Chinese. To explore the Chinese processing ability of LLMs, we focus on Chinese Text Correction, a fundamental and challenging Chinese NLP task. Specifically, we evaluate various representative LLMs on the Chinese Grammatical Error Correction (CGEC) and Chinese Spelling Check (CSC) tasks, which are two main Chinese Text Correction scenarios. Additionally, we also fine-tune LLMs for Chinese Text Correction to better observe the potential capabilities of LLMs. From extensive analyses and comparisons with previous state-of-the-art small models, we empirically find that the LLMs currently have both amazing performance and unsatisfactory behavior for Chinese Text Correction. We believe our findings will promote the landing and application of LLMs in the Chinese NLP community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑