看见推理:LLM 推理如何影响用户在事实核查任务中的信任与决策
Seeing the Reasoning: How LLM Rationales Influence User Trust and Decision-Making in Factual Verification Tasks
AI总结:
通过在线实验(N=68)操纵LLM推理的呈现格式、正确性和确定性框架,发现正确推理和确定性线索提升信任与决策采纳,不确定性线索降低,而呈现格式无显著影响。
摘要:
大型语言模型(LLMs)越来越多地在答案旁展示推理过程,将“推理”转变为用户界面元素。尽管逐步推理通常与模型性能相关,但它们在事实核查任务中如何影响用户的信任和决策仍不清楚。我们进行了一项在线研究(N=68),操纵了LLM推理过程的三个属性:呈现格式(即时 vs. 延迟 vs. 按需)、正确性(正确 vs. 错误)和确定性框架(无 vs. 确定 vs. 不确定)。我们发现,正确的推理过程和确定性线索增加了信任、决策信心和AI建议采纳,而不确定性线索则减少了这些。呈现格式没有显著影响,表明用户对推理如何揭示的敏感度低于对其可靠性的敏感度。参与者表示,他们主要使用推理来审计输出和校准信任,期望推理以逐步、自适应形式呈现并带有确定性指标。我们的工作表明,面向用户的推理如果设计不当,既可能支持决策,也可能导致信任校准错误。
英文摘要:
Large Language Models (LLMs) increasingly show reasoning rationales alongside their answers, turning "reasoning" into a user-interface element. While step-by-step rationales are typically associated with model performance, how they influence users' trust and decision-making in factual verification tasks remains unclear. We ran an online study (N=68) manipulating three properties of LLM reasoning rationales: presentation format (instant vs. delayed vs. on-demand), correctness (correct vs. incorrect), and certainty framing (none vs. certain vs. uncertain). We found that correct rationales and certainty cues increased trust, decision confidence, and AI advice adoption, whereas uncertainty cues reduced them. Presentation format did not have a significant effect, suggesting users were less sensitive to how reasoning was revealed than to its reliability. Participants indicated they use rationales to primarily audit outputs and calibrate trust, where they expected rationales in stepwise, adaptive forms with certainty indicators. Our work shows that user-facing rationales, if poorly designed, can both support decision-making yet miscalibrate trust.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。