当AI离开裁缝店：衡量LLM顾问在复杂问题解决中留下的影响
When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving
浏览论文内容
中文总结 AI 辅助
本研究通过两项预注册实验，发现LLM顾问虽提升用户信心和理解并降低努力，但撤除后仅在特定评分下提升表现，且频繁修改建议预示更好的独立能力，强调HAI评估需关注用户独立能力。
中文摘要 AI 辅助
复杂问题解决依赖于有效行动和对系统运作方式的理解。AI建议可能对这些结果的支持程度不均。两项预注册实验比较了参与者在有无LLM顾问的情况下管理模拟服装工厂的表现。跨研究发现，AI支持的参与者报告了更高的信心和理解，且付出更少的努力。在第一项研究（N=200）中，辅助增加了公司价值，但未产生可检测的预测准确性差异。撤除辅助后，当决策按重复先前选择评分时，先前受支持的参与者表现优于对照组，但在默认设置下则不然。在AI支持组内，更频繁的建议修改预测了更好的无辅助表现。在第二项研究（N=198）中，AI支持的参与者破产频率更低，且在注册分析中显示出小幅知识优势，主要与保持偿付能力相关。在AI支持组内，更频繁的建议修改预测了更高的知识水平。应用型HAI评估应评估用户的理解和独立能力，以及通过AI支持所实现的性能。
英文摘要
Complex problem solving depends on acting effectively and understanding how a system works. AI advice may support these outcomes unequally. Two preregistered experiments compared participants managing a simulated clothing factory with and without an LLM advisor. Across studies, AI-supported participants reported greater confidence and understanding with less effort. In the first study (N=200), assistance increased company value but produced no detectable prediction-accuracy difference. After withdrawal, previously supported participants outperformed controls when decisions were scored against repeating previous choices, but not default settings. Within the AI-supported group, more frequent recommendation alterations predicted better unaided performance. In the second study (N=198), AI-supported participants went bankrupt less often and showed a small knowledge advantage in the registered analysis, largely associated with remaining solvent. More frequent recommendation alterations predicted higher knowledge within the AI-supported group. Applied HAI evaluation should assess users' understanding and independent capability alongside the performance achieved with AI support.
发表机构
- Aalto University（阿尔托大学）
机构由 AI 辅助整理，请以论文原文为准。
