arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.08200cs.SE
收藏

通过可移植性方法提高软件工程实验的外部有效性

Towards Improving the External Validity of Software Engineering Experiments with Transportability Methods

Chalmers University of Technology and University of Gothenburg(查尔姆斯理工大学和哥德堡大学);The Stellenbosch Institute for Advanced Study(斯坦伦布什高级研究院);Mid Sweden University(瑞典中部大学);USI Università della Svizzera italiana(瑞士意大利语大学)

Julian Frattini, Richard Torkar, Robert Feldt, Carlo A. Furia

AI总结:

本文提出通过可移植性方法结合实验与观察数据,提升软件工程实验的外部有效性,通过模拟和场景分析展示其应用潜力及实施指南。

摘要:

受控实验是软件工程(SE)验证因果主张的核心研究方法。然而,招募能代表目标人群的样本往往困难或昂贵,限制了实验结果的外部有效性。同时,SE研究者通常能获得比实验数据更多的观察数据(例如来自仓库、问题跟踪器、日志、调查和工业流程)。可移植性方法结合实验和观察研究的数据,将实验样本的结果“移植”到更广泛、更具代表性的目标人群样本中。尽管以原则性方式结合观察和实验数据的能力可以显著促进经验SE研究,但据我们所知,可移植性方法尚未在SE中被采用。在本愿景中,我们旨在帮助实现这种采用。为此,我们介绍了可移植性方法及其前提条件,并通过模拟展示其潜力。然后,我们概述了几个SE研究场景,这些方法可能适用,例如如何有效地使用学生作为开发人员的替代品。最后,我们概述了路线图和实用指南,以支持SE研究者应用这些方法。在SE研究中采用可移植性方法可以加强受控实验的外部有效性,并帮助该领域产生更可靠、更具实践价值的结果。

英文摘要:

Controlled experiments are a core research method in software engineering (SE) for validating causal claims. However, recruiting a sample of participants that represents the intended target population is often difficult or expensive, which limits the external validity of experimental results. At the same time, SE researchers often have access to much larger amounts of observational than experimental data (e.g., from repositories, issue trackers, logs, surveys and industrial processes). Transportability methods combine these data from experimental and observational studies to "transport" results from the experimental sample to a broader, more representative sample of the target population. Although the ability to combine observational and experimental data in a principled way could substantially benefit empirical SE research, transportability methods have-to our knowledge-not been adopted in SE. In this vision, we aim to help make that adoption possible. To that end, we introduce transportability methods and their prerequisites, and demonstrate their potential through a simulation. We then outline several SE research scenarios in which these methods could apply, e.g., how to effectively use students as substitutes for developers. Finally, we outline a road map and practical guidelines to support SE researchers in applying them. Adopting transportability methods in SE research can strengthen the external validity of controlled experiments and help the field produce results that are both more reliable and more useful in practice

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑