EduBench: 一种全面的评估数据集，用于评估大型语言模型在多样化教育场景中的表现
EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios
AI总结：
本文提出EduBench数据集，用于评估大型语言模型在多样化教育场景中的表现，包含9种主要场景和4000多个教育情境，通过多维指标和人工标注验证模型性能。
摘要：
随着大型语言模型的不断发展，其在教育领域的应用仍然处于探索和优化的初级阶段。在本文中，我们通过引入首个针对教育场景的多样化基准数据集来填补这一空白，该数据集包含9种主要场景和超过4000个不同的教育情境的合成数据。为了实现全面评估，我们提出了一套多维评估指标，涵盖12个对教师和学生都重要的关键方面。我们进一步通过人工标注来确保模型生成的评估响应的有效性。此外，我们成功地在我们构建的数据集上训练了一个相对小规模的模型，并证明其在测试集上能够达到与最先进的大型模型（如DeepSeek V3、Qwen Max）相当的性能。总体而言，本工作为教育导向语言模型的发展和评估提供了实用的基础。代码和数据已发布在https://github.com/ybai-nlp/EduBench。
英文摘要：
As large language models continue to advance, their application in educational contexts remains underexplored and under-optimized. In this paper, we address this gap by introducing the first diverse benchmark tailored for educational scenarios, incorporating synthetic data containing 9 major scenarios and over 4,000 distinct educational contexts. To enable comprehensive assessment, we propose a set of multi-dimensional evaluation metrics that cover 12 critical aspects relevant to both teachers and students. We further apply human annotation to ensure the effectiveness of the model-generated evaluation responses. Additionally, we succeed to train a relatively small-scale model on our constructed dataset and demonstrate that it can achieve performance comparable to state-of-the-art large models (e.g., Deepseek V3, Qwen Max) on the test set. Overall, this work provides a practical foundation for the development and evaluation of education-oriented language models. Code and data are released at https://github.com/ybai-nlp/EduBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
