当被调查者不再关心:识别粗心回答的开始
When Respondents Don't Care Anymore: Identifying the Onset of Careless Responding
AI总结:
本文提出一种基于机器学习的方法,通过多维度数据识别粗心回答的开始,提高内部效度,提供统计保障并提供开源工具。
摘要:
行为科学中的问卷通常较长,但文献表明问卷长度是粗心回答的一个因素,较长的问卷导致参与者更可能开始粗心回答。因此,在长问卷中,大量参与者可能进行粗心回答,对内部效度构成重大威胁。本文提出一种新的方法,通过结合多个维度的测量(如不一致性和不变性)寻找粗心回答的开始(或不存在)。该方法高度灵活,基于机器学习,并提供统计保证以控制假阳性率。在模拟实验中,所提方法在识别粗心回答开始方面具有高准确性,并能有效区分认真回答和各种类型的粗心回答,即使在大量粗心回答者存在时也是如此。一个实证应用展示了如何通过识别部分粗心回答揭示粗心回答行为的新见解。此外,我们提供了免费的开源软件包
英文摘要:
Questionnaires in the behavioral sciences tend to be lengthy. However, literature suggests that survey length is a contributing factor to careless responding, with longer questionnaires yielding higher probability that participants start responding carelessly. Consequently, in long surveys a large number of participants may engage in careless responding, posing a major threat to internal validity. We propose a novel method for identifying the onset of careless responding (or an absence thereof) that searches for a changepoint in combined measurements of multiple dimensions in which carelessness may manifest, such as inconsistency and invariability. It is highly flexible, based on machine learning, and provides statistical guarantees for controlling the false positive rate. In simulation experiments, the proposed method achieves high accuracy in identifying carelessness onset and discriminates well between attentive and various types of careless responding, even when a large number of careless respondents are present. An empirical application highlights how identifying partial carelessness uncovers novel insights on careless responding behavior. Furthermore, we provide the freely available open source software package "carelessonset" to facilitate adoption by empirical researchers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。