arXiv 2510.064112025-10-07cs.CL

虚拟实验室环境中学生评估的教学目标对齐问题生成：LLM 实际对齐程度如何？

Instructional Goal-Aligned Question Generation for Student Evaluation in Virtual Lab Settings: How Closely Do LLMs Actually Align?

Department of EdPsych, University of Wisconsin-Madison（威斯康星大学麦迪逊分校教育心理学系）；Department of CS, Wichita State University（威斯康星州立大学Wichita分校计算机科学系）；Department of CSSE, Auburn University（阿伯茨罕大学计算机科学与工程系）

R. Alexander Knipper, Indrani Dey, Souvika Sarkar, Hari Narayanan, Sadhana Puntambekar, Santu Karmaker

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一个用于虚拟实验室中教学目标对齐问题生成的框架，整合目标理解、实验理解、问题分类法和 TELeR 提示控制，经 19 个开源 LLM 评估，显著提升问题质量与格式遵循度。

摘要：

虚拟实验室为动手实践、探究式科学学习提供了宝贵机会，但教师往往难以使其适应自身的教学目标。第三方材料可能与课堂需求不符，而开发定制资源既耗时又难以规模化。大型语言模型（LLM）的最新进展为解决这些局限提供了有前景的途径。在本文中，我们提出了一种新颖的教学目标对齐问题生成对齐框架，使教师能够通过自然语言交互利用 LLM 生成与模拟对齐、具有教学意义的问题。该框架整合了四个组成部分：通过教师-LLM 对话进行教学目标理解、通过知识单元和关系分析进行实验理解、用于构建认知和教学意图的问题分类法，以及用于控制提示细节的 TELeR 分类法。早期设计选择参考了一项小规模教师辅助案例研究，而我们的最终评估分析了来自 19 个开源 LLM 的 1100 多个问题。通过目标和实验理解将问题锚定在教师意图和模拟情境中，问题分类法提升了认知需求（开放式格式和关系类型将质量提高了 0.29-0.39 分），优化的 TELeR 提示增强了格式遵循度（80% 的可解析性，>90% 的遵循度）。较大的模型带来了最显著的提升：可解析性 +37.1%，遵循度 +25.7%，平均质量 +0.8 个李克特量表点。

英文摘要：

Virtual Labs offer valuable opportunities for hands-on, inquiry-based science learning, yet teachers often struggle to adapt them to fit their instructional goals. Third-party materials may not align with classroom needs, and developing custom resources can be time-consuming and difficult to scale. Recent advances in Large Language Models (LLMs) offer a promising avenue for addressing these limitations. In this paper, we introduce a novel alignment framework for instructional goal-aligned question generation, enabling teachers to leverage LLMs to produce simulation-aligned, pedagogically meaningful questions through natural language interaction. The framework integrates four components: instructional goal understanding via teacher-LLM dialogue, lab understanding via knowledge unit and relationship analysis, a question taxonomy for structuring cognitive and pedagogical intent, and the TELeR taxonomy for controlling prompt detail. Early design choices were informed by a small teacher-assisted case study, while our final evaluation analyzed over 1,100 questions from 19 open-source LLMs. With goal and lab understanding grounding questions in teacher intent and simulation context, the question taxonomy elevates cognitive demand (open-ended formats and relational types raise quality by 0.29-0.39 points), and optimized TELeR prompts enhance format adherence (80% parsability, >90% adherence). Larger models yield the strongest gains: parsability +37.1%, adherence +25.7%, and average quality +0.8 Likert points.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。