发表机构
Independent Researcher; Northeastern University; Syracuse University（独立研究员; 东北大学; Syracuse大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究基于大语言模型的教育辅导中，模型能否利用答案信息生成答案驱动的解释，用截断推理AUC评估法在三种辅导情境下评估，发现答案信息能提升模型表现，支持截断思维链审计用于诊断。
AI 中文摘要
大语言模型（LLM）辅导者常给出流畅的逐步解释，但正确且格式规范的回答不保证答案源于面向学生的问题。在实际辅导系统中，模型可能获取教师笔记等信息。我们研究此类私有答案信息能否使辅导解释由答案驱动，用截断推理AUC评估法在三种情境下评估1000个GSM8K测试问题，结果支持截断思维链审计用于数学辅导解释中的答案驱动推理诊断。
英文摘要
Large language model (LLM) tutors may have access to teacher notes, answer keys, rubrics, or retrieved solutions while producing student-facing explanations. We study whether truncated reasoning probes can distinguish direct access to such private context from answer information carried by the written explanation. Using Truncated Reasoning AUC Evaluation (TRACE), we evaluate 1000 GSM8K problems under question-only, correct answer-key, and wrong answer-key contexts. When forced-answer probes retain the private key, answer-key TRACE AUC rises from 0.375 to 0.900, and the gold answer is recoverable with no explanation at all in 998 of 1000 cases. We then introduce a context-masked replay: answer-key-generated prefixes are probed under the corresponding question-only prompt. Masking reduces 10\% prefix accuracy from 0.997 to 0.126 and median AUC from 0.900 to 0.375, nearly matching question-only values of 0.113 and 0.375. On 746 pairs where both explanations end correctly, the masked mean AUC difference is $-0.0086$ with a 95\% bootstrap interval spanning zero. Wrong keys still account for 272 of 387 incorrect final responses, showing that private artifacts can influence outputs even when early-prefix evidence disappears after masking. These results establish context masking as necessary for attributing early answer availability to an explanation rather than its hidden input.
