In the era of large language models (LLMs), in-context learning (ICL) stands out as an effective prompting strategy that explores LLMs' potency across various tasks. However, applying LLMs to grammatical error correction (GEC) is still a challenging task. In this paper, we propose a novel ungrammatical-syntax-based in-context example selection strategy for GEC. Specifically, we measure similarity of sentences based on their syntactic structures with diverse algorithms, and identify optimal ICL examples sharing the most similar ill-formed syntax to the test input. Additionally, we carry out a two-stage process to further improve the quality of selection results. On benchmark English GEC datasets, empirical results show that our proposed ungrammatical-syntax-based strategies outperform commonly-used word-matching or semantics-based methods with multiple LLMs. This indicates that for a syntax-oriented task like GEC, paying more attention to syntactic information can effectively boost LLMs' performance. Our code will be publicly available after the publication of this paper.
Large Language Model probabilities cannot reliably distinguish between possible and impossible language
Universitat Autònoma de Barcelona(巴塞罗那自治大学);Institució Catalana de Recerca i Estudis Avançats (ICREA)(加泰罗尼亚高等研究院);EcoVadis(EcoVadis 公司);Humboldt-Universität zu Berlin(柏林洪堡大学)
GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Soochow University(苏州大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);University of Science and Technology of China(中国科学技术大学)
Language models align with human judgments on key grammatical constructions
Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University(哈佛大学坎普ner自然与人工智能研究所);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Georgia Tech(佐治亚理工学院);Massachusetts Institute of Technology(麻省理工学院)
Gated Tree Cross-Attention for Checkpoint-Compatible Syntax Injection in Decoder-Only LLMs
College of Biomedical Engineering & Instrument Science, Zhejiang University(浙江大学生物医学工程与仪器科学学院);Department of Language Science and Technology, The Hong Kong Polytechnic University(香港理工大学语言科学与技术系)
How Large Language Models Get Stuck: Early structure with persistent errors
Department of Statistics(统计学系);University of Connecticut, Storrs(康涅狄格大学斯托尔分校);Department of Linguistics(语言学系);Department of Psychological Sciences(心理学科学系)