构建基于LLM模拟初学者的支架对话数据
Building Scaffolding Dialogue Data with LLM-Simulated Novices
AI总结：
本文提出SimInstruct工具，通过LLM模拟初学者生成支架对话数据，提升教学质量并发现GPT-4o的局限性。
摘要：
高质量的多轮教学对话对于开发支持教学、学习和决策的AI系统至关重要。这些对话通常涉及支架过程——专家通过问题、反馈和逐步指导来支持初学者的思维。然而，由于记录时的隐私问题和求助的脆弱性，此类数据稀缺。我们提出了SimInstruct，一种可扩展的专家在环工具，用于收集支架对话。以教学发展辅导为例，SimInstruct通过LLM模拟初学者教师，变化其教学挑战和LLM的人格特质，同时人专家提供多轮反馈、推理和教学支持。这种设计使能够创建真实、教学丰富的对话，而无需真实初学者参与者。我们的结果表明，人格特质，如外向性和内向性，显著影响专家如何参与。与真实指导录音相比，SimInstruct对话在教学相关性和认知深度上表现相当。专家还报告该过程令人投入和反思，提高了数据质量和自身专业洞察。我们进一步微调了一个LLaMA模型作为专家模型，使用增强的数据集，其在教学质量上优于GPT-4o。我们的分析突出了GPT-4o在弱反思提问、过度使用通用表扬、语气贬低以及过度建议初学者的局限性。
英文摘要：
High-quality, multi-turn instructional dialogues between novices and experts are essential for developing AI systems that support teaching, learning, and decision-making. These dialogues often involve scaffolding -- the process by which an expert supports a novice's thinking through questions, feedback, and step-by-step guidance. However, such data are scarce due to privacy concerns in recording and the vulnerability inherent in help-seeking. We present SimInstruct, a scalable, expert-in-the-loop tool for collecting scaffolding dialogues. Using teaching development coaching as an example domain, SimInstruct simulates novice instructors via LLMs, varying their teaching challenges and LLM's persona traits, while human experts provide multi-turn feedback, reasoning, and instructional support. This design enables the creation of realistic, pedagogically rich dialogues without requiring real novice participants. Our results reveal that persona traits, such as extroversion and introversion, meaningfully influence how experts engage. Compared to real mentoring recordings, SimInstruct dialogues demonstrate comparable pedagogical relevance and cognitive depth. Experts also reported the process as engaging and reflective, improving both data quality and their own professional insight. We further fine-tuned a LLaMA model to be an expert model using the augmented dataset, which outperformed GPT-4o in instructional quality. Our analysis highlights GPT-4o's limitations in weak reflective questioning, overuse of generic praise, a condescending tone, and a tendency to overwhelm novices with excessive suggestions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
