Dr.Academy：评估大语言模型教育提问能力的基准
Dr.Academy: A Benchmark for Evaluating Questioning Capability in Education for Large Language Models
AI总结：
该基准通过Anderson和Krathwohl分类法评估LLMs的教育提问能力，发现GPT-4在通用、人文和科学课程中表现优异，Claude2擅长跨学科教学，且自动评分与人类评价一致。
摘要：
教师在传授知识和引导学习者方面至关重要，而大语言模型（LLMs）作为潜在教育者的角色正成为一个重要的研究领域。认识到LLMs生成教育内容的能力可以推动自动化和个性化学习的进步。尽管LLMs已在理解和问题解决技能方面得到测试，但其教学能力在很大程度上仍未得到探索。在教学中，提问是一项关键技能，能够引导学生分析、评估和综合核心概念与原理。因此，我们的研究引入了一个基准，通过评估LLMs生成的教育问题来评估其作为教师的教育提问能力，利用Anderson和Krathwohl的分类法，涵盖通用、单学科和跨学科领域。我们将焦点从LLMs作为学习者转移到LLMs作为教育者，通过引导它们生成问题来评估其教学能力。我们应用四个指标，包括相关性、覆盖度、代表性和一致性，来评估LLMs输出的教育质量。我们的结果表明，GPT-4在教授通用、人文和科学课程方面展现出显著潜力；Claude2似乎更适合作为跨学科教师。此外，自动评分与人类观点一致。
英文摘要：
Teachers are important to imparting knowledge and guiding learners, and the role of large language models (LLMs) as potential educators is emerging as an important area of study. Recognizing LLMs' capability to generate educational content can lead to advances in automated and personalized learning. While LLMs have been tested for their comprehension and problem-solving skills, their capability in teaching remains largely unexplored. In teaching, questioning is a key skill that guides students to analyze, evaluate, and synthesize core concepts and principles. Therefore, our research introduces a benchmark to evaluate the questioning capability in education as a teacher of LLMs through evaluating their generated educational questions, utilizing Anderson and Krathwohl's taxonomy across general, monodisciplinary, and interdisciplinary domains. We shift the focus from LLMs as learners to LLMs as educators, assessing their teaching capability through guiding them to generate questions. We apply four metrics, including relevance, coverage, representativeness, and consistency, to evaluate the educational quality of LLMs' outputs. Our results indicate that GPT-4 demonstrates significant potential in teaching general, humanities, and science courses; Claude2 appears more apt as an interdisciplinary teacher. Furthermore, the automatic scores align with human perspectives.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
