指导来源至关重要:来自 AI、专家或一组分析师的指导如何影响视觉数据准备与分析
Guidance Source Matters: How Guidance from AI, Expert, or a Group of Analysts Impacts Visual Data Preparation and Analysis
AI总结:
本研究探讨了 AI、专家、分析师群体及无归属四种指导来源对用户数据准备与分析的影响,发现指导来源显著影响用户的使用方式和感知,AI 指导虽带来更高收益但也引发更多后悔。
摘要:
生成式 AI 的进展推动了 AI 驱动的工具(如 co-pilots 和助手)的发展,以提供更好的指导,特别是在数据分析期间。然而,关于指导的研究尚未考察提供指导的来源的感知有效性,以及该来源对用户感知和使用指导的影响。我们询问用户是否认为所有指导来源都是平等的,特别关注三个来源:(i) AI,(ii) 人类专家,和 (iii) 一组人类分析师。作为基准,我们考虑第四个来源,(iv) 无归属指导,其中提供指导时不归属任何来源,从而能够隔离并与特定来源指导的效果进行比较。我们设计了一项五条件被试间研究,四个指导来源各对应一个条件,外加一个额外的 (v) 无指导条件,作为基线来评估任何类型指导的影响。我们将研究置于一个定制的数据准备和分析工具中,其中我们要求用户从陌生数据集中选择相关属性以供业务报告参考。根据分配的条件,用户可以请求指导,系统随后以属性建议的形式提供指导。为确保内部有效性,我们控制了各来源条件间的指导质量。通过多项使用和感知指标,我们对五个预注册假设进行了统计检验,并报告了额外的分析。我们发现,指导来源对用户很重要,但方式并不符合传统认知。例如,尽管收到相似质量的指导,用户在分析的不同阶段以不同方式使用指导,包括表达不同程度的后悔。值得注意的是,AI 条件下的用户报告了更高的任务后收益和后悔。
英文摘要:
The progress in generative AI has fueled AI-powered tools like co-pilots and assistants to provision better guidance, particularly during data analysis. However, research on guidance has not yet examined the perceived efficacy of the source from which guidance is offered and the impact of this source on the user's perception and usage of guidance. We ask whether users perceive all guidance sources as equal, with particular interest in three sources: (i) AI, (ii) human expert, and (iii) a group of human analysts. As a benchmark, we consider a fourth source, (iv) unattributed guidance, where guidance is provided without attribution to any source, enabling isolation of and comparison with the effects of source-specific guidance. We design a five-condition between-subjects study, with one condition for each of the four guidance sources and an additional (v) no-guidance condition, which serves as a baseline to evaluate the influence of any kind of guidance. We situate our study in a custom data preparation and analysis tool wherein we task users to select relevant attributes from an unfamiliar dataset to inform a business report. Depending on the assigned condition, users can request guidance, which the system then provides in the form of attribute suggestions. To ensure internal validity, we control for the quality of guidance across source-conditions. Through several metrics of usage and perception, we statistically test five preregistered hypotheses and report on additional analysis. We find that the source of guidance matters to users, but not in a manner that matches received wisdom. For instance, users utilize guidance differently at various stages of analysis, including expressing varying levels of regret, despite receiving guidance of similar quality. Notably, users in the AI condition reported both higher post-task benefit and regret.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。