HSKBenchmark: 通过课程调优建模和评估大语言模型中的中文第二语言习得
HSKBenchmark: Modeling and Benchmarking Chinese Second Language Acquisition in Large Language Models through Curriculum Tuning
AI总结：
HSKBenchmark通过课程调优构建首个中文SLA大语言模型基准，涵盖HSK 3-6级，提供教学文本、合成指令和评估系统，验证LLMs在写作和语法上的表现。
摘要：
语言习得对于揭示人类语言智能的本质至关重要，最近已成为提高大语言模型（LLMs）可解释性的一种有前途的视角。然而，进行需要控制人类学习者语言输入的实验在伦理和实践中都是不可行的。这给语言习得建模的可验证性和可扩展性带来了挑战，特别是在中文第二语言习得（SLA）方面。尽管LLMs提供了一种可控且可重复的替代方案，但仍然缺乏一个系统性的基准来支持分阶段建模和评估。在本文中，我们提出了HSKBenchmark，这是首个针对LLMs在中文SLA中的分阶段建模和写作评估的基准。它涵盖了HSK 3至6级，并包含6.76百万词的原版教材，16K个合成指令样本，30个测试主题以及一个以语言学为基础的评估系统。为了模拟人类学习轨迹，我们引入了一个课程调优框架，该框架从初级到高级水平训练模型。创建了一个评估系统来检查基于级别的语法覆盖、写作错误、词汇和句法复杂性以及整体评分。我们还构建了HSKAgent，该模型在10,000个学习者作品上进行了微调。广泛的实验结果表明，HSKBenchmark不仅有效地建模了中文SLA，还为LLMs的动态写作评估提供了可靠的基准。我们的微调LLMs在写作表现上与高级人类学习者相当，并表现出人类般的习得特征。HSKBenchmark、HSKAgent和检查点作为基础工具和资源，有望为未来语言习得建模和LLMs可解释性研究铺平道路。代码和数据可在：https://github.com/CharlesYang030/HSKB上公开获取。
英文摘要：
Language acquisition is vital to revealing the nature of human language intelligence and has recently emerged as a promising perspective for improving the interpretability of large language models (LLMs). However, it is ethically and practically infeasible to conduct experiments that require controlling human learners' language inputs. This poses challenges for the verifiability and scalability of language acquisition modeling, particularly in Chinese second language acquisition (SLA). While LLMs provide a controllable and reproducible alternative, a systematic benchmark to support phase-wise modeling and assessment is still lacking. In this paper, we present HSKBenchmark, the first benchmark for staged modeling and writing assessment of LLMs in Chinese SLA. It covers HSK levels 3 to 6 and includes authentic textbooks with 6.76 million tokens, 16K synthetic instruction samples, 30 test topics, and a linguistically grounded evaluation system. To simulate human learning trajectories, we introduce a curriculum-tuning framework that trains models from beginner to advanced levels. An evaluation system is created to examine level-based grammar coverage, writing errors, lexical and syntactic complexity, and holistic scoring. We also build HSKAgent, fine-tuned on 10K learner compositions. Extensive experimental results demonstrate that HSKBenchmark not only models Chinese SLA effectively, but also serves as a reliable benchmark for dynamic writing assessment in LLMs. Our fine-tuned LLMs have writing performance on par with advanced human learners and exhibit human-like acquisition characteristics. The HSKBenchmark, HSKAgent, and checkpoints serve as foundational tools and resources, with the potential to pave the way for future research on language acquisition modeling and LLMs interpretability. Code and data are publicly available at: https://github.com/CharlesYang030/HSKB.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
