评分员间信度和一致性在软件工程扎根理论研究中的应用
Applying Inter-rater Reliability and Agreement in Grounded Theory Studies in Software Engineering
摘要:
近年来,应用扎根理论对经验型软件工程进行定性研究的现象越来越多。扎根理论(GT)是一种基于理论采样,编码,常数比较,记忆和饱和度作为主要特征,从定性数据中归纳迭代地发展理论的技术。大型或有争议的GT研究可能涉及多个研究人员的协作编码,这需要一种严格性和共识,而单个编码人员则不需要。虽然许多定性研究人员拒绝定量措施,赞成其他定性标准,但许多其他人致力于通过评分员间可靠性(IRR)和/或评分员间协议(IRA)技术来衡量共识,以发展对所研究现象的共同理解。然而,在GT的迭代过程中,没有关于如何以及何时应用IRR/IRA的具体指导方针,因此多年来研究人员一直使用临时方法。本文提出了一个过程,系统地应用IRR/IRA GT的研究,满足迭代性质的定性研究方法,这是支持以前的系统文献综述应用IRR/RA GT研究在软件工程。这一过程允许研究人员逐步产生一个理论,同时确保对支持它的结构达成共识,从而提高定性研究的严谨性。这种形式化有助于研究人员在编码过程中使用不同的评分者时将IRR/IRA应用于GT研究。衡量评分者之间的共识可以促进研究的可沟通性、透明度、自反性、可复制性和可信度。
英文摘要:
In recent years, the qualitative research on empirical software engineering that applies Grounded Theory is increasing. Grounded Theory (GT) is a technique for developing theory inductively e iteratively from qualitative data based on theoretical sampling, coding, constant comparison, memoing, and saturation, as main characteristics. Large or controversial GT studies may involve multiple researchers in collaborative coding, which requires a kind of rigor and consensus that an individual coder does not. Although many qualitative researchers reject quantitative measures in favor of other qualitative criteria, many others are committed to measuring consensus through Inter-Rater Reliability (IRR) and/or Inter-Rater Agreement (IRA) techniques to develop a shared understanding of the phenomenon being studied. However, there are no specific guidelines about how and when to apply IRR/IRA during the iterative process of GT, so researchers have been using ad hoc methods for years. This paper presents a process for systematically applying IRR/IRA in GT studies that meets the iterative nature of this qualitative research method, which is supported by a previous systematic literature review on applying IRR/RA in GT studies in software engineering. This process allows researchers to incrementally generate a theory while ensuring consensus on the constructs that support it and, thus, improving the rigor of qualitative research. This formalization helps researchers to apply IRR/IRA to GT studies when various raters are involved in coding. Measuring consensus among raters promotes communicability, transparency, reflexivity, replicability, and trustworthiness of the research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。