学习上下文很重要：在基于LLM的教学设计中测量和诊断个性化差距
Learning Context Matters: Measuring and Diagnosing Personalization Gaps in LLM-Based Instructional Design
AI总结：
本文提出了一种框架，用于测量和诊断学习上下文对基于LLM教学系统的影响，发现提供学习上下文能改善LLM的教学策略，但尚未实现可靠的个性化。
摘要：
生成式AI在教育中的应用近年来加速发展，大型语言模型（LLM）越来越多地被整合到学习环境中，以提供增强学习者参与度和知识保留的个性化支持。然而，真正的个性化支持需要访问有意义的学习上下文（LC），即学习者是谁、他们试图理解什么以及他们如何与材料互动。在本文中，我们提出了一种框架，用于测量和诊断LC如何影响基于LLM的辅导系统中的教学策略选择。使用心理测量学为基础的合成学习上下文和以教学为基础的决策空间，我们比较了LLM在无上下文和有上下文条件下的教学决策，并量化其与学科专家的教学生态判断的一致性。我们的结果表明，尽管提供LC会引发系统、可测量的变化，使LLM策略更接近学科专家策略，但仍然存在显著的不一致。为了诊断这种不一致，我们引入了一种相关性-影响分析，揭示了哪些学习者特征被关注、被忽略或在LLM教学决策中被错误地影响。这种分析与学科专家合作进行，表明LC在LLM教学规划中起着实质性作用，但并不能可靠地引起教学上合适的个性化。我们的结果使基于上下文的LLM系统能够得到原则性的评估，并为通过学习者特征优先级排序、教学模型调整和LC工程来改进个性化提供基础。
英文摘要：
The adoption of generative AI in education has accelerated dramatically in recent years, with Large Language Models (LLMs) increasingly integrated into learning environments in the hope of providing personalized support that enhances learner engagement and knowledge retention. However, truly personalized support requires access to meaningful Learning Context (LC) regarding who the learner is, what they are trying to understand, and how they are engaging with the material. In this paper, we present a framework for measuring and diagnosing how the LC influences instructional strategy selection in LLM-based tutoring systems. Using psychometrically grounded synthetic learning contexts and a pedagogically grounded decision space, we compare LLM instructional decisions in context-blind and context-aware conditions and quantify their alignment with the pedagogical judgments of subject matter experts. Our results show that, while providing the LC induces systematic, measurable changes in instructional decisions that move LLM policies closer to the subject matter expert policy, substantial misalignment remains. To diagnose this misalignment, we introduce a relevance-impact analysis that reveals which learner characteristics are attended to, ignored, or spuriously influential in LLM instructional decision-making. This analysis, conducted in collaboration with subject matter experts, demonstrates that LC materially shapes LLM instructional planning but does not reliably induce pedagogically appropriate personalization. Our results enable principled evaluation of context-aware LLM systems and provide a foundation for improving personalization through learner characteristic prioritization, pedagogical model tuning, and LC engineering.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
