arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.26522cs.HCcs.ET
收藏

通过较弱的LLM塑造人类-生成AI伙伴关系中的可信度判断:从交易记忆视角看AI素养

Shaping Credibility Judgments in Human-GenAI Partnership via Weaker LLMs: A Transactive Memory Perspective on AI Literacy

The Pennsylvania State University(宾夕法尼亚州立大学)

Md Touhidul Islam, Mahir Akgun, Syed Billah

AI总结:

研究通过交易记忆视角探讨学生与生成AI合作中可信度判断的塑造,采用较弱LLM设计实验,发现反思优先组学生对生成AI的依赖显著降低。

摘要:

生成式AI(GenAI)在高等教育中日益作为知识伙伴被使用,提高了对强调AI素养实践(如评估输出可信度和维持人类问责)的教学设计的需求。现有AI素养框架更关注学习者应做什么,而非这些实践在常规学生-GenAI合作中的实施方式。本文通过将学生-GenAI互动视为交易记忆伙伴关系,其中可信度调节依赖和验证。为使此过程在课程中可视化,使用了较弱的大型语言模型(LLM):足够小以在大多数学生电脑上运行,足够有帮助以支持学习,但又不足够强大以至于去除验证的必要性。在一项本科生STEM课程中,学生被随机分配到三种条件中的一个,重复活动包括反思优先(先思考,再咨询AI)、验证要求(使用AI,然后评估输出)或对照(无限制使用)。学生在三个时间点完成交易记忆调查(N=42)。权重可信度随条件变化。ANCOVA控制基线可信度显示中期有条件效应,F(2,38)=4.02,p=0.026,部分eta平方=0.175,且干预后效应更强,F(2,38)=5.48,p=0.008,部分eta平方=0.224;调整均值最低的是反思优先组,中间是验证要求组,最高是对照组。专业性和协调性的平行分析不显著。这些发现表明,工作流程排序、有意使用较弱LLM以及嵌入作业指示中的问责提示可以重新校准学生在GenAI使用中的可信度判断,其中反思优先组产生最强的依赖下降。

英文摘要:

Generative AI (GenAI) is increasingly used as a knowledge partner in higher education, raising the need for instructional designs that emphasize AI literacy practices such as evaluating output credibility and maintaining human accountability. Existing AI literacy frameworks focus more on what learners should do than on how these practices are enacted in routine student-GenAI collaboration. We address this gap by framing student-GenAI interaction as a transactive memory partnership, where credibility regulates reliance and verification. To make this process visible during coursework, we used a weaker large language model (LLM): small enough to run on most students' computers during class, helpful enough to support learning, but not so capable that it removes the need for verification. In an undergraduate STEM course, students were randomly assigned to one of three conditions across repeated activities: reflection-first (think first, then consult AI), verification-required (use AI, then evaluate the output), or control (unrestricted use). Students completed a transactive memory survey at three time points (N = 42). Weighted credibility diverged by condition over time. ANCOVA controlling for baseline credibility showed a condition effect at mid-semester, F(2, 38) = 4.02, p = .026, partial eta squared = .175, and a stronger effect at post-intervention, F(2, 38) = 5.48, p = .008, partial eta squared = .224; adjusted means were lowest in reflection-first, intermediate in verification-required, and highest in control. Parallel analyses of specialization and coordination were not significant. These findings suggest that workflow sequencing, deliberate use of weaker LLMs, and accountability cues embedded in assignment instructions can recalibrate students' credibility judgments in GenAI use, with reflection-first producing the strongest downward shift in reliance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑