BEA 2025 共享任务中的 BD：用于 AI 导师回复中教学错误识别与定位的 MPNet 集成方法
BD at BEA 2025 Shared Task: MPNet Ensembles for Pedagogical Mistake Identification and Localization in AI Tutor Responses
AI总结：
本文提出基于 MPNet 微调与 10 折分组交叉验证及硬投票集成的系统，用于 AI 导师回复中的教学错误识别与定位，在 BEA 2025 共享任务两个赛道分别取得约 0.7110 和 0.5543 的宏 F1 分数。
摘要：
我们介绍 Team BD 提交给 BEA 2025 共享任务（AI 驱动的导师教学能力评估）的系统，参与 Track 1（错误识别）和 Track 2（错误定位）。两个赛道都涉及对教育对话中导师回复进行三分类——判断导师是否正确识别学生的错误（Track 1），以及导师是否准确定位错误的位置（Track 2）。我们的系统基于 MPNet 构建，这是一种基于 Transformer 的语言模型，结合了 BERT 和 XLNet 的预训练优势。我们使用类别加权交叉熵损失在任务数据上对 MPNet 进行微调，以处理类别不平衡问题，并利用分组交叉验证（10 折）来最大化利用有限数据，同时避免训练集和验证集之间的对话重叠。然后，我们对每一折中表现最好的模型进行硬投票集成，通过组合多个分类器来提高鲁棒性和泛化能力。我们的方法在两个赛道上都取得了强劲的结果，在官方测试集上，错误识别的精确匹配宏 F1 分数约为 0.7110，错误定位约为 0.5543。我们提供了对系统性能的全面分析，包括用于解释分类器行为的混淆矩阵和 t-SNE 可视化，以及带有示例的常见错误分类体系。我们希望我们基于集成的方法和发现能为在教育对话场景中设计可靠的导师回复评估系统提供有用的见解。
英文摘要：
We present Team BD's submission to the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors, under Track 1 (Mistake Identification) and Track 2 (Mistake Location). Both tracks involve three-class classification of tutor responses in educational dialogues - determining if a tutor correctly recognizes a student's mistake (Track 1) and whether the tutor pinpoints the mistake's location (Track 2). Our system is built on MPNet, a Transformer-based language model that combines BERT and XLNet's pre-training advantages. We fine-tuned MPNet on the task data using a class-weighted cross-entropy loss to handle class imbalance, and leveraged grouped cross-validation (10 folds) to maximize the use of limited data while avoiding dialogue overlap between training and validation. We then performed a hard-voting ensemble of the best models from each fold, which improves robustness and generalization by combining multiple classifiers. Our approach achieved strong results on both tracks, with exact-match macro-F1 scores of approximately 0.7110 for Mistake Identification and 0.5543 for Mistake Location on the official test set. We include comprehensive analysis of our system's performance, including confusion matrices and t-SNE visualizations to interpret classifier behavior, as well as a taxonomy of common errors with examples. We hope our ensemble-based approach and findings provide useful insights for designing reliable tutor response evaluation systems in educational dialogue settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
