GRADE: 面向AI导师的通用推理感知对话评估
GRADE: Generalizable Reasoning-Aware Dialogue Evaluation for AI Tutors
AI总结：
提出GRADE框架，通过系统比较五种开源模型的零样本推理、LoRA微调、合成增强及思维链推理等配置，证明精心选择的LoRA流水线在关键教学维度上可媲美专有系统。
摘要：
评估AI导师的回应需要超越事实正确性：导师必须识别错误、定位错误、提供指导并提出可行的后续步骤。我们提出GRADE，一项针对学生-导师对话中教学能力评估的开源模型系统研究。基于BEA 2025 TutorMind设置，我们评估了五种语言模型、零样本推理、LoRA微调、合成增强、思维链推理以及单任务与多任务公式化配置下的120种配置。Gemma3-12B在单任务评估中表现最佳，而8位精度的Gemma3-27B在多任务预测中更可靠。我们发现，增强有助于那些在原始数据上表现不佳的模型，验证虽然成本更高但增益有限，思维链推理对合成数据生成比直接分类更有用。我们进一步表明，在结构化分类目标上进行LoRA微调会干扰思考模式下的指令遵循行为，使生成偏离所需的评估格式。碳分析显示，模型选择和推理模式显著影响排放。总体而言，GRADE表明，精心选择的开源LoRA流水线可以在关键教学维度上匹配或超越专有和基于集成系统的性能，代码和数据可在https://github.com/pvbgeek/GRADE获取。
英文摘要：
Evaluating AI tutor responses requires more than factual correctness: tutors must identify mistakes, locate errors, provide guidance, and offer actionable next steps. We present GRADE, a systematic study of open-source models for pedagogical ability assessment in student-tutor dialogues. Building on the BEA 2025 TutorMind setting, we evaluate 120 configurations across five language models, zero-shot inference, LoRA fine-tuning, synthetic augmentation, CoT+Reasoning, and single-task versus multitask formulations. Gemma3-12B performs best for single-task evaluation, while Gemma3-27B in 8-bit precision is more reliable for multitask prediction. We find that augmentation helps models that struggle with the original data, verification adds limited gains despite higher cost, and CoT+Reasoning is more useful for synthetic data generation than direct classification. We further show that LoRA fine-tuning on structured classification objectives interferes with instruction-following behavior under thinking mode, redirecting generation away from the required evaluation format. Carbon analysis shows that model choice and reasoning mode substantially affect emissions. Overall, GRADE shows that carefully selected open-source LoRA pipelines can match or surpass proprietary and ensemble-based systems on key pedagogical dimensions, with code and data available at https://github.com/pvbgeek/GRADE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
