生成式AI建议采纳中的元认知与信心动态
Metacognition and Confidence Dynamics in Advice Taking from Generative AI
AI总结:
本研究通过两项文本生成实验,探究人与GenAI互动中前瞻性、回顾性信心对建议寻求与依赖的影响,发现采纳GenAI建议可因果提升信心,但用户易因输出冗长忽略核查,揭示了信心在人机互动中的核心作用。
摘要:
生成式人工智能(GenAI)可在各类任务中为人类提供辅助,但其有效性关键取决于用户能否评估GenAI输出的准确性以及自身的专业能力。本研究探讨了对自身和GenAI的信心如何影响寻求并依赖GenAI建议的决策(即“前瞻性信心”),以及采纳建议又如何反过来塑造这种信心(即“回顾性信心”)。\n研究采用涉及文本生成的新颖范式,参与者需制定活动计划,他们既可以主动向GenAI请求建议(研究1;样本量N=200),也会被随机分配接收建议(研究2;样本量N=300),且可选择依赖或忽略这些建议。研究1中,建议请求行为与对GenAI更高的前瞻性信心、对自身更低的信心相关。寻求建议的参与者对GenAI的回顾性信心有所提升,而拒绝建议的参与者对自身的信心有所增强。研究2的随机分配结果显示,接触建议会提升对GenAI和自身的信心,表明采纳GenAI建议对回顾性信心具有因果提升作用。\n上述结果在建议依赖程度上也得到了印证——建议依赖以GenAI建议与参与者回答的文本相似度来衡量,依赖程度越高,对GenAI和自身的回顾性信心均越高。关键的是,选择获取/依赖建议的参与者给出的回答更详细(这可能源于GenAI输出的冗长性),但他们并未彻底核查输出内容,遗漏了关键信息。这些发现凸显了信心在人与GenAI互动中的核心作用,这种信心既受关于自身的先验信念与AI可靠性的影响,也受情境化的建议接触所塑造。
英文摘要:
Generative Artificial Intelligence (GenAI) can aid humans in a wide range of tasks, but its effectiveness critically depends on users being able to evaluate the accuracy of GenAI outputs and their own expertise. Here we asked how confidence in self and GenAI contributes to decisions to seek and rely on advice from GenAI ('prospective confidence'), and how advice-taking in turn shapes this confidence ('retrospective confidence'). In a novel paradigm involving text generation, participants formulated plans for events, and could request advice from a GenAI (Study 1; N=200) or were randomly assigned to receive advice (Study 2; N=300), which they could rely on or ignore. Advice requests in Study 1 were related to higher prospective confidence in GenAI and lower confidence in self. Advice-seekers showed increased retrospective confidence in GenAI, while those who declined advice showed increased confidence in self. Random assignment in Study 2 revealed that advice exposure increases confidence in GenAI and in self, suggesting that GenAI advice-taking causally boosts retrospective confidence. These results were mirrored in advice reliance, operationalised as the textual similarity between GenAI advice and participants' responses, with reliance associated with increased retrospective confidence in both GenAI and self. Critically, participants who chose to obtain/rely on advice provided more detailed responses (likely due to the output's verbosity), but failed to check the output thoroughly, missing key information. These findings underscore a key role for confidence in interactions with GenAI, shaped by both prior beliefs about oneself and the reliability of AI, and context-dependent exposure to advice.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。