arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.19296cs.HCstat.AP
收藏

按需辅导异质性效应估计的因果框架

A Causal Framework for Estimating Heterogeneous Effects of On-Demand Tutoring

Cornell University(康奈尔大学);Carnegie Mellon University(卡内基梅隆大学);Eedi Labs(Eedi实验室);Massachusetts Institute of Technology(麻省理工学院)

Kirk Vanacore, Danielle R Thomas, Digory Smith, Bibi Groot, Justin Reich, Rene Kizilcec

AI总结:

提出一个可扩展的因果推断框架,结合深度知识追踪和因果森林,估计自适应学习系统中按需人工辅导的即时效果,发现辅导能提升后续题目正确率约4个百分点,且效果存在显著异质性。

摘要:

本文介绍了一个可扩展的因果推断框架,用于估计自适应学习系统中嵌入的按需人工辅导的即时、会话级效果。由于学生在遇到困难时寻求帮助,传统评估会受到自我选择和时间变化知识状态的混淆。我们通过将原则性分析样本构建与深度知识追踪(DKT)相结合来估计潜在掌握度,然后使用因果森林进行双重稳健估计,从而解决了这些挑战。将该框架应用于超过5000次中学数学辅导会话,我们发现请求人工辅导可使下一题正确率提高约4个百分点,并使后续遇到的技能准确率提高约3个百分点,这表明辅导效果在知识组件间存在近端迁移。该效果对多种模型规范和潜在未测量混杂因素具有稳健性。值得注意的是,这些效果在会话和学生之间表现出显著异质性,会话级效果估计范围从-20.25个百分点到+19.91个百分点。我们的后续分析表明,典型的行为指标(如学生发言时间)与高影响会话并不一致相关。此外,对于先前掌握度较低的学生,处理效果更大;对于低社会经济地位(SES)的学生,效果略小。该框架为评估和持续改进按需人工辅导提供了一个严格、实用的模板,并可直接应用于新兴的人工智能辅导系统。

英文摘要:

This paper introduces a scalable causal inference framework for estimating the immediate, session-level effects of on-demand human tutoring embedded within adaptive learning systems. Because students seek assistance at moments of difficulty, conventional evaluation is confounded by self-selection and time-varying knowledge states. We address these challenges by integrating principled analytic sample construction with Deep Knowledge Tracing (DKT) to estimate latent mastery, followed by doubly robust estimation using Causal Forests. Applying this framework to over 5,000 middle-school mathematics tutoring sessions, we find that requesting human tutoring increases next-problem correctness by approximately 4 percentage points and accuracy on the subsequent skill encountered by approximately 3 percentage points, suggesting that the effects of tutoring have proximal transfer across knowledge components. This effect is robust to various forms of model specification and potential unmeasured confounders. Notably, these effects exhibit significant heterogeneity across sessions and students, with session-level effect estimates ranging from $-20.25pp$ to $+19.91pp$. Our follow-up analyses suggest that typical behavioral indicators, such as student talk time, do not consistently correlate with high-impact sessions. Furthermore, treatment effects are larger for students with lower prior mastery and slightly smaller for low-SES students. This framework offers a rigorous, practical template for the evaluation and continuous improvement of on-demand human tutoring, with direct applications for emerging AI tutoring systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑