EduEval: 一个用于评估大语言模型在中文教育中表现的分层认知基准
EduEval: A Hierarchical Cognitive Benchmark for Evaluating Large Language Models in Chinese Education
AI总结：
EduEval通过分层认知基准评估大语言模型在中文教育中的表现，揭示其在事实性任务与创造性任务上的差异，并发现开源模型在复杂教育推理上的优势。
摘要：
大语言模型（LLMs）在教育应用中展现出显著潜力。然而，其未经审视的部署可能对教育标准构成风险，凸显了严格评估的必要性。我们介绍了EduEval，一个全面的分层基准，用于评估LLMs在中文K-12教育中的表现。该基准有三个关键贡献：（1）认知框架：我们提出了EduAbility Taxonomy，将Bloom的分类法和Webb的深度知识分类统一起来，以六个认知维度（记忆、理解、应用、推理、创造和伦理）组织任务。（2）真实性：我们的基准集整合了真实考试题目、课堂对话、学生作文和专家设计的提示，以反映真实的教育挑战；（3）规模：EduEval包含24种不同的任务类型，超过11000个问题，涵盖小学到高中各层次。我们评估了14个领先的LLMs，在零样本和少量样本设置下，发现虽然模型在事实性任务上表现良好，但在课堂对话分类和创造性内容生成方面存在困难。有趣的是，一些开源模型在复杂的教育推理任务上表现优于专有系统。少量样本提示在不同认知维度上显示了不同的有效性，表明不同的教育目标需要定制的方法。这些发现为开发专门优化于多样化中文教育任务的LLMs提供了有针对性的基准度量。
英文摘要：
Large language models (LLMs) demonstrate significant potential for educational applications. However, their unscrutinized deployment poses risks to educational standards, underscoring the need for rigorous evaluation. We introduce EduEval, a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education. This benchmark makes three key contributions: (1) Cognitive Framework: We propose the EduAbility Taxonomy, which unifies Bloom's Taxonomy and Webb's Depth of Knowledge to organize tasks across six cognitive dimensions including Memorization, Understanding, Application, Reasoning, Creativity, and Ethics. (2) Authenticity: Our benchmark integrates real exam questions, classroom conversation, student essays, and expert-designed prompts to reflect genuine educational challenges; (3) Scale: EduEval comprises 24 distinct task types with over 11,000 questions spanning primary to high school levels. We evaluate 14 leading LLMs under both zero-shot and few-shot settings, revealing that while models perform well on factual tasks, they struggle with classroom dialogue classification and exhibit inconsistent results in creative content generation. Interestingly, several open source models outperform proprietary systems on complex educational reasoning. Few-shot prompting shows varying effectiveness across cognitive dimensions, suggesting that different educational objectives require tailored approaches. These findings provide targeted benchmarking metrics for developing LLMs specifically optimized for diverse Chinese educational tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
