Simulation studies are commonly used in methodological research for the empirical evaluation of data analysis methods. They generate artificial data sets under specified mechanisms and compare the performance of methods across conditions. However, simulation repetitions do not always produce valid outputs, e.g., due to non-convergence or other algorithmic failures. This phenomenon complicates the interpretation of results, especially when its occurrence differs between methods and conditions. Despite the potentially serious consequences of such "missingness", quantitative data on its prevalence and specific guidance on how to deal with it are currently limited. To this end, we reviewed 482 simulation studies published in various methodological journals and systematically assessed the prevalence and handling of missingness. We found that only 23% (111/482) of the reviewed simulation studies mention missingness, with even fewer reporting frequency (92/482 = 19%) or how it was handled (67/482 = 14%). We propose a classification of missingness and possible solutions. We give various recommendations, most notably to always quantify and report missingness, even if none was observed, to align missingness handling with study goals, and to share code and data for reproduction and reanalysis. Using a case study on publication bias adjustment methods, we illustrate common pitfalls and solutions.
Handling Missingness and Censoring in Dirichlet Models
University of Pretoria（比勒陀利亚大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（国家理论与计算科学研究所）；San José State University（圣何塞州立大学）；University of Catania（卡塔尼亚大学）
Compatibility of Missing Data Handling Methods across the Stages of Producing Clinical Prediction Models
University of Manchester（曼彻斯特大学）；University of Cambridge（剑桥大学）；The University of Liverpool（利物浦大学）；Microsoft Research（微软研究院）；Manchester University Hospital NHS foundation Trust（曼彻斯特大学医院NHS基金会信托）；Health Data Research UK（英国健康数据研究）；University of Birmingham（伯明翰大学）
Parametric MMD Estimation with Missing Values: Robustness to Missingness and Data Model Misspecification
CNRS, LPSM, Sorbonne Université, Université Paris Cité（法国国家科学研究中心，统计与概率实验室，索邦大学，巴黎西岱大学）；Research Institute for Statistics and Information Science, University of Geneva（日内瓦大学统计与信息科学研究院）
Handling incomplete outcomes and covariates in cluster-randomized trials: doubly-robust estimation, efficiency considerations, and sensitivity analysis
University of Michigan（密歇根大学）；Yale School of Public Health（耶鲁公共卫生学院）；Harvard Pilgrim Health Care Institute（哈佛皮尔格里姆健康护理研究所）；Harvard Medical School（哈佛医学院）；Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）
Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity
Ludwig-Maximilians University Munich（路德维希-马克西米利安慕尼黑大学）；Centre for Integrated Data and Epidemiological Research（综合数据与流行病学研究中心）；UMIT - University for Health Sciences, Medical Informatics and Technology（UMIT - 健康科学、医学信息与技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
A Review of Methods and Practices for Missing Data in Sequential Multiple Assignment Randomized Trials (SMARTs): An Ancillary Study of a Scoping Review
Duke University（杜克大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Rethinking the handling of method failure in comparison studies
LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Faculty of Medicine, LMU Munich（慕尼黑大学医学院）；Department of Statistics, LMU Munich（慕尼黑大学统计系）；MRC Clinical Trials Unit, UCL（伦敦大学学院医学研究委员会临床试验单元）
Handling mild outliers and unobserved values in compositional datasets using finite mixtures of mean-parametrised Dirichlet models
University of Pretoria（比勒陀利亚大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（理论与计算科学国家研究所）；San José State University（圣何塞州立大学）；University of Catania（卡塔尼亚大学）
