生成式人工智能驱动的智能辅导系统的教学法驱动评估
Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systems
AI总结：
本文综述生成式AI驱动的智能辅导系统评估现状，指出缺乏教学法驱动基准的挑战，并提出三个基于学习科学的实用研究方向，以建立公平统一的评估方法。
摘要：
人工智能教育（AIED）这一跨学科研究领域在开发智能辅导系统（ITSs）方面有着悠久的历史，其通过整合技术进步、教育理论和认知心理学的见解而不断发展。生成式人工智能（GenAI）模型的显著成功加速了基于大语言模型（LLM）的智能辅导系统的发展，这些系统有潜力模仿类似人类的、教学法丰富且认知要求高的辅导过程。然而，由于缺乏可靠、普遍接受且教学法驱动的评估框架和基准，这些系统的进展和影响在很大程度上仍难以追踪。大多数现有的基于教育对话的智能辅导系统评估依赖于主观协议和非标准化基准，导致结果不一致且泛化能力有限。在本工作中，我们从主流智能辅导系统开发中退一步，提供了全面的最先进评估实践综述，并通过来自细致且关怀的AIED研究的真实案例研究突出了相关挑战。最后，基于先前跨学科AIED研究的见解，我们提出了三个实用、可行且具有理论依据的研究方向，这些方向植根于学习科学原理，旨在为智能辅导系统建立公平、统一且可扩展的评估方法。
英文摘要：
The interdisciplinary research domain of Artificial Intelligence in Education (AIED) has a long history of developing Intelligent Tutoring Systems (ITSs) by integrating insights from technological advancements, educational theories, and cognitive psychology. The remarkable success of generative AI (GenAI) models has accelerated the development of large language model (LLM)-powered ITSs, which have potential to imitate human-like, pedagogically rich, and cognitively demanding tutoring. However, the progress and impact of these systems remain largely untraceable due to the absence of reliable, universally accepted, and pedagogy-driven evaluation frameworks and benchmarks. Most existing educational dialogue-based ITS evaluations rely on subjective protocols and non-standardized benchmarks, leading to inconsistencies and limited generalizability. In this work, we take a step back from mainstream ITS development and provide comprehensive state-of-the-art evaluation practices, highlighting associated challenges through real-world case studies from careful and caring AIED research. Finally, building on insights from previous interdisciplinary AIED research, we propose three practical, feasible, and theoretically grounded research directions, rooted in learning science principles and aimed at establishing fair, unified, and scalable evaluation methodologies for ITSs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
