FoundationalASSIST: 一个用于基础知识追踪和LLM教学基础的教育数据集
FoundationalASSIST: An Educational Dataset for Foundational Knowledge Tracing and Pedagogical Grounding of LLMs
AI总结：
FoundationalASSIST 是首个为研究LLM在教育中的应用而设计的英文教育数据集，提供完整问题文本、学生实际回答和对齐标准，用于评估LLM在知识追踪和教学基础方面的性能。
摘要：
大语言模型能否理解学生如何学习？随着LLM被用于自适应测试和个性化教学，这个问题变得紧迫——然而，我们无法用现有的资源来回答这个问题。当前的教育数据集只提供问题标识符和二元正确性标签，使这些数据对能够用自然语言推理的LLM来说显得模糊。我们通过FoundationalASSIST填补这一空白，这是第一个提供完整信息的英文教育数据集，用于研究LLM在教育中的应用：完整的问题文本、学生实际的回答（不仅仅是对或错）、记录学生选择的错误答案以及与共同核心K-12标准的对齐。这些来自5000名学生的170万次互动使研究方向变得可能，从微调学生模型到分析误解模式。为了展示数据集的效用，我们评估了四个前沿模型（GPT-OSS-120B、Llama-3.3-70B、Qwen3-Next-80B变体）在两个互补的任务家族上：知识追踪，测试LLM能否预测学生在问题上的表现，以及学生将给出的准确答案；以及教学基础，测试LLM是否理解使评估项目有效的属性。我们的评估揭示了当前LLM能力的重大差距。每个模型在知识追踪上 barely 仅达到基线水平。所有模型在项目区分上都低于随机猜测，表明LLM不理解什么使一个问题比另一个更具诊断性。模型在判断相对难度方面显示出一定的能力（最高68.6%），但这种部分成功仅突显了其他方面的差距。这些结果表明，在LLM能够可靠地支持大规模个性化学习之前，仍需取得重大进展。我们发布FoundationalASSIST以支持在这些基础挑战上的进步。
英文摘要：
Can Large Language Models understand how students learn? As LLMs are deployed for adaptive testing and personalized tutoring, this question becomes urgent -- yet we cannot answer it with existing resources. Current educational datasets provide only question identifiers and binary correctness labels, rendering them opaque to LLMs that reason in natural language. We address this gap with FoundationalASSIST, the first English educational dataset providing the complete information needed for research on LLMs in education: full question text, actual student responses (not just right/wrong), records of which wrong answers students chose, and alignment to Common Core K-12 standards. These 1.7 million interactions from 5,000 students enable research directions that were previously impossible to pursue, from fine-tuning student models to analyzing misconception patterns. To demonstrate the dataset's utility, we evaluate four frontier models (GPT-OSS-120B, Llama-3.3-70B, Qwen3-Next-80B variants) on two complementary task families: Knowledge Tracing, testing whether LLMs can predict student performance on questions, and the exact answer a student will give; and \textbf{Pedagogical Grounding}, testing whether LLMs understand the properties that make assessment items effective. Our evaluation reveals significant gaps in current LLM capabilities. Every model barely achieves a trivial baseline on knowledge tracing. All models fall below random chance on item discrimination, indicating that LLMs do not understand what makes one problem more diagnostic than another. Models do show competence at judging relative difficulty (up to 68.6%), but this partial success only highlights the gaps elsewhere. These results establish that substantial advances are needed before LLMs can reliably support personalized learning at scale. We release FoundationalASSIST to support progress on these foundational challenges.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
