arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

ExplainRoute:非答案提供型编程 tutor 的部署前审计框架

ExplainRoute: A Pre-Deployment Audit Framework for Non-Answer-Giving Programming Tutors

Yiming Gai, Yingying Zhang, Xuefei Huang

arXiv 2609.03470首次发表:更新:

AI 中文总结

本文提出 ExplainRoute 框架,用于审计非答案提供型编程 tutor,评估发现其自适应路由未优于固定策略,仅贡献验证的审计协议与相关边界发现。

AI 中文摘要

编程 tutor 应支持学习者自主解释,而非直接提供模型答案。本文提出 ExplainRoute,这是一个针对非答案提供型编程 tutor 的部署前审计框架。给定一行代码和一段学习者解释,该框架会估计解释状态,并从两种受限响应中选择一种:费曼式自我解释提示或苏格拉底式支架。该框架通过机器可验证契约公开其状态、策略、引用代码片段及信息泄露风险。与仅按流畅度对 tutor 排名的基准不同,ExplainRoute 在课堂部署前审计信息边界、响应极性、失败闭合性及学习者解释可见性的价值。我们在包含1770对的 SelfCode 语料库上,采用代码分组拆分进行离线评估,其中11个未触碰的保留组共包含443对。评估对比了直接答案、固定开放自我解释、固定苏格拉底式支架、自适应路由及无状态自适应消融。所有教学条件下的契约有效性均达到100%。自适应路由在60.5%的记录上与冻结参考规则匹配,状态宏F1为0.238(开放:0.229;苏格拉底式:0.246),无可靠自适应优势。独立语言模型裁判对自适应响应的评分为4.516/5,优于无状态消融(2.819/5),但略低于固定开放自我解释(4.598/5)和苏格拉底式支架(4.658/5)。对分层40行子集的盲法评分评估确认,可见的学习者解释提升了信息价值,而自适应路由未优于固定策略。本研究的贡献是经过验证的审计协议及一项边界发现,而非关于学习、保持或因果教学有效性提升的证据。

英文摘要

Programming tutors should support learners' own explanations rather than immediately providing model answers. We present ExplainRoute, a pre-deployment audit framework for non-answer-giving programming tutors. Given a code line and a learner explanation, it estimates the explanation state and selects one of two bounded responses: a Feynman-style self-explanation prompt or a Socratic scaffold. The framework exposes its state, strategy, cited code fragment, and leakage risk through a machine-checkable contract. Unlike benchmarks that rank tutors by fluency alone, ExplainRoute audits information boundaries, response polarity, failure closure, and the value of learner-explanation visibility before classroom deployment. We evaluate it offline on the 1,770-pair SelfCode corpus using a code-group split, with 443 pairs reserved in 11 untouched holdout groups. The evaluation compares direct answers, fixed open self-explanation, fixed Socratic scaffolding, adaptive routing, and an adaptive no-state ablation. Contract validity reaches 100% for all pedagogical conditions. Adaptive routing matches the frozen reference rule on 60.5% of records, with state macro-F1 of 0.238 (Open: 0.229; Socratic: 0.246), showing no reliable adaptive advantage. An independent language-model judge scores adaptive responses 4.516/5, outperforming the no-state ablation (2.819/5) but slightly below fixed open self-explanation (4.598/5) and Socratic scaffolding (4.658/5). A blinded rubric evaluation on a stratified 40-row subset confirms that visible learner explanations improve information value while adaptive routing does not outperform fixed strategies. The contribution is a validated audit protocol and a boundary finding, rather than evidence of improved learning, retention, or causal instructional effectiveness.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑