arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.03202cs.CLcs.AI
收藏

ChatLang-8:一种基于 LLM 的语法错误纠正合成数据生成框架

ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction

Atommerce;Upstage AI;Korea University(高丽大学)

Jeiyoon Park, Chanjun Park, Heuiseok Lim

AI总结:

本文提出含主题、语法选择与评估模块的 LLM 合成数据框架,并构建含 100 万句对的 ChatLang-8 数据集,显著改善 GEC 数据分布与模型性能。

摘要:

我们探索并提升了 LLM 为语法错误纠正(GEC)生成数据的能力。当 LLM 仅仅生成平行句时,其模式过于简单,作为语料库缺乏价值。为解决这一问题,我们提出了一个自动化框架,包含 Subject Selector、Grammar Selector、Prompt Manager 和 Evaluator。此外,我们引入了一个用于 GEC 任务的新数据集,名为 ChatLang-8,涵盖 8 种主题名词和 23 种语法。它由 100 万个具有类人语法错误的句对组成。实验表明,与现有 GEC 数据集相比,ChatLang-8 展现出更均匀的模式构成。此外,我们观察到,使用 ChatLang-8 替代现有 GEC 数据集时,模型性能有所提升。实验结果表明,我们的框架和 ChatLang-8 是增强 ChatGPT 数据生成能力的宝贵资源。

英文摘要:

We explore and improve the capabilities of LLMs to generate data for grammatical error correction (GEC). When merely producing parallel sentences, their patterns are too simplistic to be valuable as a corpus. To address this issue, we propose an automated framework that includes a Subject Selector, Grammar Selector, Prompt Manager, and Evaluator. Additionally, we introduce a new dataset for GEC tasks, named ChatLang-8, which encompasses eight types of subject nouns and 23 types of grammar. It consists of 1 million pairs featuring human-like grammatical errors. Our experiments reveal that ChatLang-8 exhibits a more uniform pattern composition compared to existing GEC datasets. Furthermore, we observe improved model performance when using ChatLang-8 instead of existing GEC datasets. The experimental results suggest that our framework and ChatLang-8 are valuable resources for enhancing ChatGPT's data generation capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑