MSA在BEA 2025共享任务中的应用：面向LLM数学辅导教师多维评估的感知分歧指令微调
MSA at BEA 2025 Shared Task: Disagreement-Aware Instruction Tuning for Multi-Dimensional Evaluation of LLMs as Math Tutors
AI总结：
本文提出MSA-MathEval系统，采用统一指令微调和感知分歧的集成推理，在BEA 2025共享任务中实现LLM数学辅导多维评估，并在提供指导等维度取得领先排名。
摘要：
我们提出了MSA-MathEval，这是我们对BEA 2025共享任务的提交，该任务旨在评估AI导师在四个教学维度上的回应：错误识别、错误定位、提供指导和可操作性。我们的方法采用统一的训练流程，在无需任何任务特定架构更改的情况下，跨所有轨道微调单个指令微调语言模型。为了提高预测可靠性，我们引入了一种感知分歧的集成推理策略，该策略增强了对少数标签的覆盖。我们的系统在所有轨道上均取得了强劲性能，在提供指导方面排名第一，在可操作性方面排名第三，在错误识别和错误定位方面均排名第四。这些结果证明了可扩展的指令微调和分歧驱动建模在稳健、多维评估LLM作为教育导师方面的有效性。
英文摘要：
We present MSA-MathEval, our submission to the BEA 2025 Shared Task on evaluating AI tutor responses across four instructional dimensions: Mistake Identification, Mistake Location, Providing Guidance, and Actionability. Our approach uses a unified training pipeline to fine-tune a single instruction-tuned language model across all tracks, without any task-specific architectural changes. To improve prediction reliability, we introduce a disagreement-aware ensemble inference strategy that enhances coverage of minority labels. Our system achieves strong performance across all tracks, ranking 1st in Providing Guidance, 3rd in Actionability, and 4th in both Mistake Identification and Mistake Location. These results demonstrate the effectiveness of scalable instruction tuning and disagreement-driven modeling for robust, multi-dimensional evaluation of LLMs as educational tutors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
