In designing external validation studies of clinical prediction models, contemporary sample size calculation methods are based on the frequentist inferential paradigm. One of the widely reported metrics of model performance is net benefit (NB), and the relevance of conventional inference around NB as a measure of clinical utility is doubtful. Value of Information methodology quantifies the consequences of uncertainty in terms of its impact on clinical utility of decisions. We introduce the expected value of sample information (EVSI) for validation as the expected gain in NB from conducting an external validation study of a given size. We propose algorithms for EVSI computation, and in a case study demonstrate how EVSI changes as a function of the amount of current information and future study's sample size. Value of Information methodology provides a decision-theoretic lens to the process of planning a validation study of a risk prediction model and can complement conventional methods when designing such studies.
A Probabilistic Domain-knowledge Framework for Nosocomial Infection Risk Estimation of Communicable Viral Diseases in Healthcare Personnel: A Case Study for COVID-19
Development and Validation of a Dynamic Kidney Failure Prediction Model based on Deep Learning: A Real-World Study with External Validation
Peking University(北京大学);National University of Singapore(新加坡国立大学);Peking University First Hospital(北京大学第一医院);Chinese Academy of Medical Sciences(中国医学科学院);Ningbo Yinzhou No.2 Hospital(宁波市鄞州第二医院);Yinzhou District Center for Disease Control and Prevention(鄞州区疾病预防控制中心)
Bayesian sample size calculations for external validation studies of risk prediction models
Faculty of Pharmaceutical Sciences, The University of British Columbia(不列颠哥伦比亚大学药学院);Department of Statistics, The University of British Columbia(不列颠哥伦比亚大学统计系);British Columbia Centre For Disease Control(不列颠哥伦比亚疾病控制中心);CAPHRI Care and Public Health Research Institute, Maastricht University(马斯特里赫特大学CAPHRI健康与公共护理研究所);Department of Development and Regeneration, KU Leuven(荷语鲁汶大学发展与再生系);Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham(伯明翰大学健康科学学院应用健康科学系);National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre(英国国家卫生保健研究卓越创新中心伯明翰生物医学研究中心)
Value-of-Information Analysis for External Validation of Risk Prediction Models in Multicenter Studies and Systematic Reviews
Maastricht University(马斯特里赫特大学);KU Leuven(荷语鲁汶大学);Maastricht University Medical Centre+ (MUMC+)(马斯特里赫特大学医学中心+);Memorial Sloan Kettering Cancer Center(纪念斯隆-凯特琳癌症中心);University Medical Center Utrecht, Utrecht University(乌得勒支大学医学中心,乌得勒支大学);The University of British Columbia(不列颠哥伦比亚大学)