Towards Improving the External Validity of Software Engineering Experiments with Transportability Methods
Chalmers University of Technology and University of Gothenburg(查尔姆斯理工大学和哥德堡大学);The Stellenbosch Institute for Advanced Study(斯坦伦布什高级研究院);Mid Sweden University(瑞典中部大学);USI Università della Svizzera italiana(瑞士意大利语大学)
Julian Frattini, Richard Torkar, Robert Feldt, Carlo A. Furia
Controlled experiments are a core research method in software engineering (SE) for validating causal claims. However, recruiting a sample of participants that represents the intended target population is often difficult or expensive, which limits the external validity of experimental results. At the same time, SE researchers often have access to much larger amounts of observational than experimental data (e.g., from repositories, issue trackers, logs, surveys and industrial processes). Transportability methods combine these data from experimental and observational studies to "transport" results from the experimental sample to a broader, more representative sample of the target population. Although the ability to combine observational and experimental data in a principled way could substantially benefit empirical SE research, transportability methods have-to our knowledge-not been adopted in SE. In this vision, we aim to help make that adoption possible. To that end, we introduce transportability methods and their prerequisites, and demonstrate their potential through a simulation. We then outline several SE research scenarios in which these methods could apply, e.g., how to effectively use students as substitutes for developers. Finally, we outline a road map and practical guidelines to support SE researchers in applying them. Adopting transportability methods in SE research can strengthen the external validity of controlled experiments and help the field produce results that are both more reliable and more useful in practice
Replications, Revisions, and Reanalyses: Managing Empirical Evidence in Software Engineering
Chalmers University of Technology and Gothenburg University(查尔姆斯理工大学和哥德堡大学);Netlight Consulting GmbH(Netlight咨询公司);fortiss GmbH(fortiss公司);Software Engineering Research Lab, Blekinge Institute of Technology(布莱金厄理工学院软件工程研究实验室);Mid Sweden University(瑞典中部大学);Stellenbosch Institute for Advanced Study (STIAS)(斯坦伦布什高等研究院)
Mitigating Omitted Variable Bias in Empirical Software Engineering
Software Institute, USI Università della Svizzera italiana(瑞士意大利大学软件研究所);Chalmers University of Technology and University of Gothenburg(查尔姆斯理工大学和哥德堡大学);Stellenbosch Institute for Advanced Study (STIAS)(斯坦伦布什高级研究院)
The Software Infrastructure Attitude Scale (SIAS): A Questionnaire Instrument for Measuring Professionals' Attitudes Toward Technical and Sociotechnical Infrastructure
Dalhousie University(达尔豪斯大学);University of Helsinki(赫尔辛基大学);Northwestern University(西北大学);University of Calgary(卡尔加里大学);Mahidol University(玛希隆大学);New York Institute of Technology(纽约理工学院);King Saud University(沙特国王大学);NADI, University of Namur(纳穆尔大学 NADI);Shinshu University(信州大学);Heidelberg University(海德堡大学);University of Bayreuth(拜罗伊特大学);Bilkent University(比尔肯特大学)
The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys
University of Calgary(卡尔加里大学);University of Hawaii at Manoa(夏威夷大学马诺阿分校);University of Bari(巴里大学);Universidade Federal Rural de Pernambuco (UFRPE)(伯南布哥联邦农村大学);Radboud University(拉德堡德大学)
Property-Driven Synthetic Data Engineering for Data-Scarce Software Systems: Reflections from the Breast Cancer Domain
University of Bergamo(博洛尼亚大学);University of Toronto(多伦多大学);Humanitas Clinical and Research Center, IRCCS(人类itas临床与研究中心);ASST Papa Giovanni XXIII(ASST圣若望二十三世医院)
Generative AI and Empirical Software Engineering: A Paradigm Shift
School of Computing(计算学院);Information Systems Singapore Management University Singapore, Singapore(信息系统新加坡管理大学新加坡);Department of Computer Science University of Victoria Victoria, Canada(计算机科学系维多利亚大学加拿大)
Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills
Chengdu Institute of Computer Applications, Chinese Academy of Sciences(中国科学院成都计算机应用研究所);University of Chinese Academy of Sciences(中国科学院大学);Institute of Multidisciplinary Research for Advanced Materials (IMRAM), Tohoku University(东北大学先进材料多学科研究所);Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology(深圳先进技术研究院人工智能研究所)
College of Computer Science and Technology and the State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学计算机科学与技术学院和区块链与数据安全全国重点实验室);Hong Kong University of Science and Technology(香港科技大学);CISPA Helmholtz Center for Information Security(CISPA赫尔姆霍兹信息安全中心)