AI 中文总结
研究可视化实验中基于绩效的激励措施的影响，通过两项众包研究，分别考察其对低级感知任务和推理任务的作用，发现激励对任务表现无影响，提示此类激励可能未必达预期效果，引发对激励研究设计的思考。
AI 中文摘要
可视化研究中的一个长期问题是确定特定数据集的哪种视觉编码对用户执行特定任务最有效，通常通过对照实验来识别不同编码的相对有效性。然而，设计实验时在实验设置上有很多临时决策，如是否包含训练模块、是否给参与者提供基于绩效的激励等，且对于如何做这些决策指导有限。本文研究了货币奖励这一关键设计决策的影响，通过两项众包研究调查激励对低级感知任务和推理任务的影响，结果发现激励对两项任务的表现均无影响，这表明基于绩效的财务激励可能并不总是产生预期效果，呼吁反思激励研究的设计方式。
英文摘要
A perennial research question in visualization involves identifying which visual encodings for a particular dataset are most effective for users in performing a specific task. The relative effectiveness of the different encodings are commonly identified through controlled experiments. However, designing an experiment involves making many, often ad hoc, decisions about the experimental setup such as whether to include a training module, whether to provide performance-based incentives to participants, etc. Yet, there is limited guidance on how these decisions should be made, and we do not fully understand the impact of these subjective decisions on empirical results. In this paper, we investigate the impact of one such key design decision: monetary rewards. Specifically, we ask: does providing or not providing participants with performance-based financial incentives affect the results and the conclusions that we draw from visualization studies? We conducted two crowdsourced studies investigating the impact of incentives on (i) a low-level, perceptual task (perception of correlations in scatterplots or parallel coordinate plots), and (ii) a task involving reasoning (decision-making based on a weather forecast represented as intervals or density plots). In each of these studies, we manipulate both the visual representation and the presence of incentives as between-subject conditions. We expected to find no effect of incentives on the perceptual task, but to see an effect for the decision-making task. However, we found no effect on task performance in either study. While these are results of only two studies and should be replicated, they suggest that performance-based financial incentives may not always have the intended effect on participants that we presumed, and calls for a reflection of how incentivized studies should be designed.
