Clinical prediction models (CPMs) are used to predict clinically relevant outcomes or events. Typically, prognostic CPMs are derived to predict the risk of a single future outcome. However, with rising emphasis on the prediction of multi-morbidity, there is growing need for CPMs to simultaneously predict risks for each of multiple future outcomes. A common approach to multi-outcome risk prediction is to derive a CPM for each outcome separately, then multiply the predicted risks. This approach is only valid if the outcomes are conditionally independent given the covariates, and it fails to exploit the potential relationships between the outcomes. This paper outlines several approaches that could be used to develop prognostic CPMs for multiple outcomes. We consider four methods, ranging in complexity and assumed conditional independence assumptions: namely, probabilistic classifier chain, multinomial logistic regression, multivariate logistic regression, and a Bayesian probit model. These are compared with methods that rely on conditional independence: separate univariate CPMs and stacked regression. Employing a simulation study and real-world example via the MIMIC-III database, we illustrate that CPMs for joint risk prediction of multiple outcomes should only be derived using methods that model the residual correlation between outcomes. In such a situation, our results suggest that probabilistic classification chains, multinomial logistic regression or the Bayesian probit model are all appropriate choices. We call into question the development of CPMs for each outcome in isolation when multiple correlated or structurally related outcomes are of interest and recommend more holistic risk prediction.
In many clinical trials treatments need to be repeatedly applied as diseases relapse frequently after remission over a long period of time (e.g., 35 weeks). Most research in statistics focuses on the overall trial design, such as sample size and power calculation, or on the data analysis after trials are completed. Little is done to improve the efficiency of trial monitoring, such as early termination of trials due to futility. The challenge faced in such trial monitoring is mostly caused by the need to properly model repeated outcomes from patients. We propose a Bayesian trial monitoring sche…
Clinical outcome prediction from high-dimensional data is problematic in the common setting where there is only a relatively small number of samples. The imbalance causes data overfitting, and outcome prediction becomes computationally expensive or even impossible. We propose a Bayesian outcome prediction method that can be applied to data of arbitrary dimension d, from 2 outcome classes, and reduces overfitting without any approximations at parameter level. This is achieved by avoiding numerical integration or approximation, and solving the Bayesian integrals analytically. We thereby reduce t…
External information, such as prior information or expert opinions, can play an important role in the design, analysis and interpretation of clinical trials. However, little attention has been devoted thus far to incorporating external information in clinical trials with binary outcomes, perhaps due to the perception that binary outcomes can be treated as normally-distributed outcomes by using normal approximations. In this paper we show that these two types of clinical trials could behave differently, and that special care is needed for the analysis of clinical trials with binary outcomes. In…
Global Blood Therapeutics（Global Blood Therapeutics（全球血液治疗公司））
The approval success rate of drug candidates is very low with the majority of failure due to safety and efficacy. Increasingly available high dimensional information on targets, drug molecules and indications provides an opportunity for ML methods to integrate multiple data modalities and better predict clinically promising drug targets. Notably, drug targets with human genetics evidence are shown to have better odds to succeed. However, a recent tensor factorization-based approach found that additional information on targets and indications might not necessarily improve the predictive accurac…
The primary analysis in two-arm clinical trials usually involves inference on a scalar treatment effect parameter; e.g., depending on the outcome, the difference of treatment-specific means, risk difference, risk ratio, or odds ratio. Most clinical trials are monitored for the possibility of early stopping. Because ordinarily the outcome on any given subject can be ascertained only after some time lag, at the time of an interim analysis, among the subjects already enrolled, the outcome is known for only a subset and is effectively censored for those who have not been enrolled sufficiently long…
Missing data is unavoidable in longitudinal clinical trials, and outcomes are not always normally distributed. In the presence of outliers or heavy-tailed distributions, the conventional multiple imputation with the mixed model with repeated measures analysis of the average treatment effect (ATE) based on the multivariate normal assumption may produce bias and power loss. Control-based imputation (CBI) is an approach for evaluating the treatment effect under the assumption that participants in both the test and control groups with missing outcome data have a similar outcome profile as those wi…
In many randomized clinical trials of therapeutics for COVID-19, the primary outcome is an ordinal categorical variable, and interest focuses on the odds ratio (active agent vs. control) under the assumption of a proportional odds model. Although at the final analysis the outcome will be determined for all subjects, at an interim analysis, the status of some participants may not yet be determined, e.g., because ascertainment of the outcome may not be possible until some pre-specified follow-up time. Accordingly, the outcome from these subjects can be viewed as censored. A valid interim analysi…
The risk ratio is a popular tool for summarizing the relationship between a binary covariate and outcome, even when outcomes may be dependent. Investigations of infectious disease outcomes in cohort studies of individuals embedded within clusters -- households, villages, or small groups -- often report risk ratios. Epidemiologists have warned that risk ratios may be misleading when outcomes are contagious, but the nature and severity of this error is not well understood. In this study, we assess the epidemiologic meaning of the risk ratio when outcomes are contagious. We first give a structura…
The recent availability of routine medical data, especially in a university-clinical context, may enable the discovery of typical healthcare pathways, i.e., typical temporal sequences of clinical interventions or hospital readmissions. However, such pathways are heterogeneous in a large provider such as a university hospital, and it is important to identify similar care pathways that can still be considered typical pathways. We understand the pathway as a temporal process with possible transitions from a single initial treatment state to hospital readmission of different types, which constitut…
Hospital profiling involves a comparison of a health care provider's structure, processes of care, or outcomes to a standard, often in the form of a report card. Given the ubiquity of report cards and similar consumer ratings in contemporary American culture, it is notable that these are a relatively recent phenomenon in health care. Prior to the 1986 release of Medicare hospital outcome data, little such information was publicly available. We review the historical evolution of hospital profiling with special emphasis on outcomes; present a detailed history of cardiac surgery report cards, the…
Background: In epidemiology, causal inference and prediction modeling methodologies have been historically distinct. Directed Acyclic Graphs (DAGs) are used to model a priori causal assumptions and inform variable selection strategies for causal questions. Although tools originally designed for prediction are finding applications in causal inference, the counterpart has remained largely unexplored. The aim of this theoretical and simulation-based study is to assess the potential benefit of using DAGs in clinical risk prediction modeling. Methods and Findings: We explore how incorporating knowl…
A decomposition of Fisher's information to inform sample size for developing or updating fair and precise clinical prediction models -- Part 3: continuous outcomes
The assurance method is growing in popularity in clinical trial planning. The method involves eliciting a prior distribution for the treatment effect, and then calculating the probability that a proposed trial will produce a `successful' outcome. For normally distributed observations, uncertainty about the variance of the normal distribution also needs to be accounted for, but there is little guidance in the literature on how to elicit a distribution for a variance parameter. We present a simple elicitation method, and illustrate how the elicited distribution is incorporated within an assuranc…
Risk Assessment Instruments (RAIs) are widely used to forecast adverse outcomes in domains such as healthcare and criminal justice. RAIs are commonly trained on observational data and are optimized to predict observable outcomes rather than potential outcomes, which are the outcomes that would occur absent a particular intervention. Examples of relevant potential outcomes include whether a patient's condition would worsen without treatment or whether a defendant would recidivate if released pretrial. We illustrate how RAIs which are trained to predict observable outcomes can lead to worse deci…
In many clinical trials, outcomes of interest include binary-valued endpoints. It is not uncommon that a binary-valued outcome is dichotomized from a continuous outcome at a threshold of clinical interest. To reach the objective, common approaches include (a) fitting the generalized linear mixed model (GLMM) to the dichotomized longitudinal binary outcome and (b) imputation method (MI): imputing the missing values in the continuous outcome, dichotomizing it into a binary outcome, and then fitting the generalized linear model for the "complete" data. We conducted comprehensive simulation studie…
Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, e.g. in terms of sample size and / or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in form of extensive simulation studies, than conventional approaches. Here we present a framework for adaptive treatment and subgroup selection …
Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. (2012) proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. In this article, we propose a general framework, called Residual Weighted Learning (RWL), to i…
To improve precision of estimation and power of testing hypothesis for an unconditional treatment effect in randomized clinical trials with binary outcomes, researchers and regulatory agencies recommend using g-computation as a reliable method of covariate adjustment. However, the practical application of g-computation is hindered by the lack of an explicit robust variance formula that can be used for different unconditional treatment effects of interest. To fill this gap, we provide explicit and robust variance estimators for g-computation estimators and demonstrate through simulations that t…
The analysis and planning methods for competing risks model have been described in the literatures in recent decades, and non-inferiority clinical trials are helpful in current pharmaceutical practice. Analytical methods for non-inferiority clinical trials in the presence of competing risks were investigated by Parpia et al., who indicated that the proportional sub-distribution hazard model is appropriate in the context of biological studies. However, the analytical methods of competing risks model differ from those appropriate for analyzing non-inferiority clinical trials with a single outcom…
Impact of clinical decision support systems (cdss) on clinical outcomes and healthcare delivery in low- and middle-income countries: protocol for a systematic review and meta-analysis
ICMR National Institute for Research in Digital Health & Data Science（印度国家数字健康与数据科学研究所）；ICMR Headquarters（印度国家医学研究委员会总部）；ICMR-RMRC Bhubaneshwar（印度国家医学研究委员会比哈尔邦研究中心）
Many clinical endpoint measures, such as the number of standard drinks consumed per week or the number of days that patients stayed in the hospital, are count data with excessive zeros. However, the zero-inflated nature of such outcomes is sometimes ignored in analyses of clinical trials. This leads to biased estimates of study-level intervention effect and, consequently, a biased estimate of the overall intervention effect in a meta-analysis. The current study proposes a novel statistical approach, the Zero-inflation Bias Correction (ZIBC) method, that can account for the bias introduced when…
Copula Based Fusion of Clinical and Genomic Machine Learning Risk Scores for Breast Cancer Risk Stratification
Department of Mathematics, University of Louisiana at Lafayette（路易斯安那州立大学拉斐特分校数学系）；Department of Physical Sciences & Mathematics, West Liberty University（韦斯特里布利大学物理科学与数学系）；Department of Mathematics and Statistics, Minnesota State University（明尼苏达州立大学数学与统计学系）
In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can be incorporated using penalized splines. Penalized splines can be conveniently represented as a linear mixed effects model (LMM), where the coefficients of the spline basis functions are random effects. The LMM representation of penalized splines makes the extension to multivariate outcomes relatively straightforward. In the LMM, no effect of the quantitative covariate on the …
While the gold standard for clinical trials is to blind all parties -- participants, researchers, and evaluators -- to treatment assignment, this is not always a possibility. When some or all of the above individuals know the treatment assignment, this leaves the study open to the introduction of post-randomization biases. In the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) trial, we were presented with the potential for the unblinded clinicians administering the treatment, as well as the individuals enrolled in the study, to introduce ascertainment bias into some bu…
Personalized medicine seeks to identify the causal effect of treatment for a particular patient as opposed to a clinical population at large. Most investigators estimate such personalized treatment effects by regressing the outcome of a randomized clinical trial (RCT) on patient covariates. The realized value of the outcome may however lie far from the conditional expectation. We therefore introduce a method called Dirac Delta Regression (DDR) that estimates the entire conditional density from RCT data in order to visualize the probabilities across all possible outcome values. DDR transforms t…
TRACER: Transfer Learning based Real-time Adaptation for Clinical Evolving Risk
Duke University School of Medicine（杜克大学医学学院）；Duke AI Health（杜克AI健康）；Department of Biostatistics and Bioinformatics（生物统计学与生物信息学系）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Centre for Quantitative Medicine（定量医学中心）；Duke-NUS AI + Medical Sciences Initiative（杜克-新加坡国立大学AI+医学科学倡议）；Pre-hospital & Emergency Research Centre（急救与应急研究中心）；Health Services and Systems Research（卫生服务与系统研究）；NUS Artificial Intelligence Institute（新加坡国立大学人工智能研究所）
I develop a model of a randomized experiment with a binary intervention and a binary outcome. Potential outcomes in the intervention and control groups give rise to four types of participants. Fixing ideas such that the outcome is mortality, some participants would live regardless, others would be saved, others would be killed, and others would die regardless. These potential outcome types are not observable. However, I use the model to develop estimators of the number of participants of each type. The model relies on the randomization within the experiment and on deductive reasoning. I apply …
Selective recruitment designs preferentially recruit individuals that are estimated to be statistically informative onto a clinical trial. Individuals that are expected to contribute less information have a lower probability of recruitment. Furthermore, in an information-adaptive design recruits are allocated to treatment arms in a manner that maximises information gain. The informativeness of an individual depends on their covariate (or biomarker) values and how information is defined is a critical element of information-adaptive designs. In this paper we define and evaluate four different me…
Predictive risk scores estimating probabilities for a binary outcome on the basis of observed covariates are common across the sciences. They are frequently developed with the intent of avoiding the outcome in question by intervening in response to estimated risks. Since risk scores are usually developed in complex systems, interventions usually take the form of expert agents responding to estimated risks as they best see fit. In this case, interventions may be complex and their effects difficult to observe or infer, meaning that explicit specification of interventions in response to risk scor…
