GrammarGPT：通过监督微调探索开源大语言模型在母语中文语法错误纠正中的应用
GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning
AI总结：
本文提出 GrammarGPT，利用 ChatGPT 合成与人工标注的约千条混合数据及错误不变增强，对开源 LLM 进行指令微调，在中文语法纠错任务上以少 1200 倍数据显著超越 SOTA。
摘要：
语法错误纠正旨在自动纠正不合语法的句子。近期，一些研究已经证明了闭源大语言模型（LLM，例如 ChatGPT）在语法错误纠正方面的出色能力。然而，开源 LLM 的潜力仍未得到探索。本文介绍了开源 LLM GrammarGPT，以初步探索其在母语中文语法错误纠正中的潜力。GrammarGPT 的核心方案是利用由 ChatGPT 生成数据与人工标注数据组成的混合数据集。对于带有线索的语法错误，我们提出了一种启发式方法，通过提供这些线索来引导 ChatGPT 生成不合语法的句子。对于没有线索的语法错误，我们从公开可用的网站收集不合语法的句子，并进行人工纠正。此外，我们采用了一种错误不变增强方法，以提升模型纠正母语中文语法错误的能力。我们最终构建了约 1k 条平行数据，并利用这些数据通过指令微调对开源 LLM（例如香港中文大学（深圳）发布的 Phoenix）进行微调。实验结果表明，GrammarGPT 显著优于现有 SOTA 系统。尽管模型参数量比 SOTA 基线大 20 倍，但指令微调所需的数据量少 1200 倍，这说明了开源 LLM 在母语 CGEC 上的潜力。我们的 GrammarGPT 在 NLPCC2023 SharedTask1 中排名第 3，证明了本文方法的有效性。代码和数据可在 https://github.com/FreedomIntelligence/GrammarGPT 获取。
英文摘要：
Grammatical error correction aims to correct ungrammatical sentences automatically. Recently, some work has demonstrated the excellent capabilities of closed-source Large Language Models (LLMs, e.g., ChatGPT) in grammatical error correction. However, the potential of open-source LLMs remains unexplored. In this paper, we introduced GrammarGPT, an open-source LLM, to preliminary explore its potential for native Chinese grammatical error correction. The core recipe of GrammarGPT is to leverage the hybrid dataset of ChatGPT-generated and human-annotated. For grammatical errors with clues, we proposed a heuristic method to guide ChatGPT to generate ungrammatical sentences by providing those clues. For grammatical errors without clues, we collected ungrammatical sentences from publicly available websites and manually corrected them. In addition, we employed an error-invariant augmentation method to enhance the ability of the model to correct native Chinese grammatical errors. We ultimately constructed about 1k parallel data and utilized these data to fine-tune open-source LLMs (e.g., Phoenix, released by The Chinese University of Hong Kong, Shenzhen) with instruction tuning. The experimental results show that GrammarGPT outperforms the existing SOTA system significantly. Although model parameters are 20x larger than the SOTA baseline, the required amount of data for instruction tuning is 1200x smaller, illustrating the potential of open-source LLMs on native CGEC. Our GrammarGPT ranks $3^{rd}$ on NLPCC2023 SharedTask1, demonstrating our approach's effectiveness. The code and data are available at \url{https://github.com/FreedomIntelligence/GrammarGPT}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
