BEA 2023 教育对话中AI教师回复生成共享任务
The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues
AI总结：
该研究介绍BEA 2023首个教育对话AI教师回复生成共享任务，8支团队用多种大模型参赛，NAISTeacher系统凭GPT-3.5结合提示集成与DialogRPT排序夺冠，同时指出需更适配教育场景的评估指标。
摘要：
本文介绍了首个教育对话教师回复生成共享任务的结果。该任务旨在基准测试生成式语言模型在师生对话中作为AI教师回复学生的能力。共有8支团队参与了在CodaLab平台举办的竞赛，他们实验了多种前沿模型，包括Alpaca、Bloom、DialoGPT、DistilGPT-2、Flan-T5、GPT-2、GPT-3、GPT-4、LLaMA、OPT-2.7B和T5-base。参赛作品使用BERTScore和DialogRPT指标自动评分，排名前三的作品还基于Tack和Piech（2022）的框架进行了教学能力人工评估。在自动评估和人工评估中均位列第一的NAISTeacher系统，采用GPT-3.5生成回复，结合提示词集成策略与基于DialogRPT的对话上下文回复排序方法。尽管参赛团队取得了可观成果，结果仍凸显出需要更适配教育场景的评估指标。
英文摘要：
This paper describes the results of the first shared task on the generation of teacher responses in educational dialogues. The goal of the task was to benchmark the ability of generative language models to act as AI teachers, replying to a student in a teacher-student dialogue. Eight teams participated in the competition hosted on CodaLab. They experimented with a wide variety of state-of-the-art models, including Alpaca, Bloom, DialoGPT, DistilGPT-2, Flan-T5, GPT-2, GPT-3, GPT- 4, LLaMA, OPT-2.7B, and T5-base. Their submissions were automatically scored using BERTScore and DialogRPT metrics, and the top three among them were further manually evaluated in terms of pedagogical ability based on Tack and Piech (2022). The NAISTeacher system, which ranked first in both automated and human evaluation, generated responses with GPT-3.5 using an ensemble of prompts and a DialogRPT-based ranking of responses for given dialogue contexts. Despite the promising achievements of the participating teams, the results also highlight the need for evaluation metrics better suited to educational contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
