EssayBench：评估大语言模型在多体裁中文写作中的表现
EssayBench: Evaluating Large Language Models in Multi-Genre Chinese Essay Writing
AI总结：
本文提出多体裁中文写作基准EssayBench，包含728个真实题目及细粒度体裁化评分框架，并系统评估15个大语言模型在四类中文作文中的表现与局限。
摘要：
中文写作及其评价在教育场景中至关重要，但大语言模型（LLMs）在这一领域的能力仍远未被充分探索。现有基准通常依赖粗粒度的文本质量指标，在很大程度上忽视了中文作文在结构性与修辞性方面的复杂性，尤其是在不同体裁之间。为填补这一空白，我们提出 EssayBench，一个专门面向中文写作的多体裁基准，覆盖议论文、记叙文、说明文和描写文四大主要体裁。我们收集并精炼了共728个真实写作题目以确保真实性，并将其细致划分为开放式与限定式两类，以覆盖多样化的写作场景。为可靠地评估生成作文，我们设计了一套细粒度、按体裁区分的评分框架，对分数进行层级化聚合。我们进一步通过全面的人工一致性研究验证了该评估方案。最后，我们对15个大规模大语言模型进行了基准测试，分析其在不同体裁和指令类型下的优势与局限。借助 EssayBench，我们希望推动基于大语言模型的中文作文评价发展，并启发未来在教育场景中改进作文生成的研究。
英文摘要：
Chinese essay writing and its evaluation are critical in educational contexts, yet the capabilities of Large Language Models (LLMs) in this domain remain largely underexplored. Existing benchmarks often rely on coarse-grained text quality metrics, largely overlooking the structural and rhetorical complexities of Chinese essays, particularly across diverse genres. To address this gap, we propose \benchName, a multi-genre benchmark specifically designed for Chinese essay writing across four major genres: Argumentative, Narrative, Descriptive, and Expository. We curate and refine a total of 728 real-world prompts to ensure authenticity and meticulously categorize them into the \textit{Open-Ended} and \textit{Constrained} sets to capture diverse writing scenarios. To reliably evaluate generated essays, we develop a fine-grained, genre-specific scoring framework that hierarchically aggregates scores. We further validate our evaluation protocol through a comprehensive human agreement study. Finally, we benchmark 15 large-sized LLMs, analyzing their strengths and limitations across genres and instruction types. With \benchName, we aim to advance LLM-based Chinese essay evaluation and inspire future research on improving essay generation in educational settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
