arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.16251cs.HC
收藏

RelianceScope:一种分析学生在问题解决中依赖生成式AI聊天机器人行为的分析框架

RelianceScope: An Analytical Framework for Examining Students' Reliance on Generative AI Chatbots in Problem Solving

University of Michigan(密歇根大学);KAIST(韩国科学技术院);HKUST(香港科技大学)

Hyoungwook Jin, Minju Yoo, Jieun Han, Zixin Chen, So-Yeon Ahn, Xu Wang

AI总结:

本文提出RelianceScope框架,通过分析学生在求助和响应使用中的参与模式,揭示其依赖行为,为教育系统设计提供指导。

摘要:

生成式AI聊天机器人能够实现个性化问题解决,但有效的学习需要学生自我调节求助和使用AI生成响应的方式。研究发现,分析这两种行为的参与模式可以揭示复杂的依赖模式:例如,学生可能在求助时主动明确需求领域,但被动复制AI输出。现有研究缺乏系统工具来同时捕捉这两种行为的参与度,限制了对依赖行为的分析。本文引入RelianceScope分析框架,通过将求助和响应使用的参与模式组合,定义出九种依赖模式,并将这些模式置于知识情境视角中,考虑学生先前知识和知识组件的教学意义。该框架不规定最优的AI使用方式,而是使研究人员能够对开放性学生-AI互动中的依赖行为进行细致分析。作为示例应用,我们分析了79名大学生在网页编程课程中的聊天和代码编辑日志。结果显示,主动求助与主动响应使用相关,而依赖模式在知识掌握水平上相似。学生往往难以描述知识缺口并调整AI响应。使用我们标注的数据集作为基准,进一步证明大型语言模型可以可靠地检测求助和响应使用中的依赖行为。最后讨论了RelianceScope的启示及AI支持教育系统的设计指南。

英文摘要:

Generative AI chatbots enable personalized problem-solving, but effective learning requires students to self-regulate both how they seek help and how they use AI-generated responses. Considering engagement modes across these two actions reveals nuanced reliance patterns: for example, a student may actively engage in help-seeking by clearly specifying areas of need, yet engage passively in response-use by copying AI outputs, or vice versa. However, existing research lacks systematic tools for jointly capturing engagement across help-seeking and response-use, limiting the analysis of such reliance behaviors. We introduce RelianceScope, an analytical framework that characterizes students' reliance on chatbots during problem-solving. RelianceScope (1) operationalizes reliance into nine patterns based on combinations of engagement modes in help-seeking and response-use, and (2) situates these patterns within a knowledge-context lens that accounts for students' prior knowledge and the instructional significance of knowledge components. Rather than prescribing optimal AI use, the framework enables fine-grained analysis of reliance in open-ended student-AI interactions. As an illustrative application, we applied RelianceScope to analyze chat and code-edit logs from 79 college students in a web programming course. Results show that active help-seeking is associated with active response-use, whereas reliance patterns remain similar across knowledge mastery levels. Students often struggled to articulate their knowledge gaps and to adapt AI responses. Using our annotated dataset as a benchmark, we further demonstrate that large language models can reliably detect reliance during help-seeking and response-use. We conclude by discussing the implications of RelianceScope and the design guidelines for AI-supported educational systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑