CSTutorBench：将小型语言模型作为基于块的编程导师进行基准测试
CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming
AI总结：
研究针对K-12环境中部署语言模型的问题，引入CSTutorBench基准评估VEX VR中语言模型作导师的表现。通过对11个模型测试发现其在深层教学行为有困难，模型家族和指令调整方法对辅导质量预测性更好，特定提示修订可提分，凸显此类基准的价值。
摘要：
大语言模型越来越多地被用作人工智能导师，但在 K-12 环境中部署引发了对隐私、成本和对专有模型依赖的担忧。小型语言模型是有前景的替代方案，但为特定教育环境选择合适模型仍困难，尤其目标领域如基于块的编程在模型训练数据中大多缺失。我们引入CSTutorBench，用于评估VEX VR（基于块的机器人环境）中语言模型作为计算机科学导师的基准。它包含17个基于场景的问题，根据教学评分标准打分，通过人工参与的大语言模型评判管道评估。对11个模型（4B - 120B参数）的初步发现表明，模型在词汇和语气等表面标准上表现良好，但在更深层次的教学行为上存在困难，特别是避免答案泄露和处理学生调试历史。在样本中，模型家族和指令调整方法似乎比参数数量更能预测辅导质量。基于近期教育提示工程研究的有针对性的提示修订提高了11个模型中10个的分数。这些结果强调了在教育部署中为小型语言模型选择提供特定于上下文、基于教学的基准的价值。
英文摘要：
Large language models are increasingly explored as AI tutors, yet deploying them in K-12 settings raises concerns around privacy, cost, and reliance on proprietary models. Small language models (SLMs) offer a promising alternative, but selecting the right model for a specific educational context remains difficult, particularly when the target domain, such as block-based programming, is largely absent from model training data. We introduce CSTutorBench, a benchmark for evaluating language models as CS tutors in VEX VR, a block-based robotics environment. The benchmark comprises 17 scenario-based questions scored against a pedagogical rubric grounded in established tutoring and feedback research, with a human-in-the-loop LLM-as-judge pipeline for evaluation. Preliminary findings across 11 models (4B-120B parameters) reveal that models perform well on surface-level criteria such as vocabulary and tone but struggle with deeper pedagogical behaviors, particularly avoiding answer leakage and engaging with student debugging histories. In our sample, model family and instruction-tuning approach appear to be better predictors of tutoring quality than parameter count alone, though the small number of models limits the strength of this conclusion. A targeted prompt revision grounded in recent educational prompt engineering research improved scores for 10 of 11 models. These results underscore the value of context-specific, pedagogically grounded benchmarks for SLM selection in educational deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
