Selection Induced Contrast Estimate (SICE) Effect: An Attempt to Quantify the Impact of Some Patient Selection Criteria in Randomized Clinical Trials
Junshui Ma, Daniel J. Holder
备注：
Many tricks have been hidden in the complex Inclusion/Exclusion (I/E) criteria of clinical studies. People need to have an in-depth understanding of those I/E criteria to better understand when and how a treatment can/should be used
Defining the Inclusion/Exclusion (I/E) criteria of a trial is one of the most important steps during a trial design. Increasingly complex I/E criteria potentially create information imbalance and transparency issues between the people who design and run the trials and those who consume the information produced by the trials. In order to better understand and quantify the impact of a category of I/E criteria on observed treatment effects, a concept, named the Selection Induced Contrast Estimate (SICE) effect, is introduced and formulated in this paper. The SICE effect can exist in controlled clinical trials when treatment affects the correlation between a marker used for selection and the response of interest. This effect is demonstrated with both simulations and real clinical trial data. Although the statistical elements behind the SICE effect have been well studied, explicitly formulating and studying this effect can benefit several areas, including better transparency in I/E criteria, meta-analysis of multiple clinical trials, treatment effect interpretation in real-world medical practice, etc.
When choosing estimands and estimators in randomized clinical trials, caution is warranted as intercurrent events, such as - due to patients who switch treatment after disease progression, are often extreme. Statistical analyses may then easily lure one into making large implicit extrapolations, which often go unnoticed. We will illustrate this problem of implicit extrapolations using a real oncology case study, with a right-censored time-to-event endpoint, in which patients can cross over from the control to the experimental treatment after disease progression, for ethical reasons. We resolve…
A utility-based Bayesian population finding (BaPoFi) method was proposed by Morita and Müller (2017, Biometrics, 1355-1365) to analyze data from a randomized clinical trial with the aim of identifying good predictive baseline covariates for optimizing the target population for a future study. The approach casts the population finding process as a formal decision problem together with a flexible probability model using a random forest to define a regression mean function. BaPoFi is constructed to handle a single continuous or binary outcome variable. In this paper, we develop BaPoFi-TTE as an e…
Cluster randomized trails (CRT) have been widely employed in medical and public health research. Many clinical count outcomes, such as the number of falls in nursing homes, exhibit excessive zero values. In the presence of zero inflation, traditional power analysis methods for count data based on Poisson or negative binomial distribution may be inadequate. In this study, we present a sample size method for CRTs with zero-inflated count outcomes. It is developed based on GEE regression directly modeling the marginal mean of a ZIP outcome, which avoids the challenge of testing two intervention e…
Efforts to develop biomarker-targeted anti-cancer therapies have progressed rapidly in recent years. Six antibodies acting on programmed death ligand 1 or programmed death 1 pathways were approved in 75 cancer indications between 2015 and 2021. With efforts to expedite regulatory reviews of promising therapies, several targeted cancer therapies have been granted accelerated approval on the basis of single-arm phase II trials. And yet, in the absence of randomization, patient outcomes may not have been studied under standard of care chemotherapies for emerging biomarker subpopulations prior to …
Phase III randomized clinical trials play a monumentally critical role in the evaluation of new medical products. Because of the intrinsic nature of uncertainty embedded in our capability in assessing the efficacy of a medical product, interpretation of trial results relies on statistical principles to control the error of false positives below desirable level. The well-established statistical hypothesis testing procedure suffers from two major limitations, namely, the lack of flexibility in the thresholds to claim success and the lack of capability of controlling the total number of false pos…
University of Cambridge（剑桥大学）；Newcastle University（纽卡斯尔大学）
High-dimensional biomarkers such as genomics are increasingly being measured in randomized clinical trials. Consequently, there is a growing interest in developing methods that improve the power to detect biomarker-treatment interactions. We adapt recently proposed two-stage interaction detecting procedures in the setting of randomized clinical trials. We also propose a new stage 1 multivariate screening strategy using ridge regression to account for correlations among biomarkers. For this multivariate screening, we prove the asymptotic between-stage independence, required for family-wise erro…
Many clinical endpoint measures, such as the number of standard drinks consumed per week or the number of days that patients stayed in the hospital, are count data with excessive zeros. However, the zero-inflated nature of such outcomes is sometimes ignored in analyses of clinical trials. This leads to biased estimates of study-level intervention effect and, consequently, a biased estimate of the overall intervention effect in a meta-analysis. The current study proposes a novel statistical approach, the Zero-inflation Bias Correction (ZIBC) method, that can account for the bias introduced when…
The use of information from real world to assess the effectiveness of medical products is becoming increasingly popular and more acceptable by regulatory agencies. According to a strategic real-world evidence framework published by U.S. Food and Drug Administration, a hybrid randomized controlled trial that augments internal control arm with real-world data is a pragmatic approach worth more attention. In this paper, we aim to improve on existing matching designs for such a hybrid randomized controlled trial. In particular, we propose to match the entire concurrent randomized clinical trial (R…
The analysis of randomized trials with time-to-event endpoints is nearly always plagued by the problem of censoring. As the censoring mechanism is usually unknown, analyses typically employ the assumption of non-informative censoring. While this assumption usually becomes more plausible as more baseline covariates are being adjusted for, such adjustment also raises concerns. Pre-specification of which covariates will be adjusted for (and how) is difficult, thus prompting the use of data-driven variable selection procedures, which may impede valid inferences to be drawn. The adjustment for cova…
The primary outcome of Randomized clinical Trials (RCTs) are typically dichotomous, continuous, multivariate continuous, or time-to-event. However, what if this outcome is unstructured, e.g., a list of variables of mixed types, longitudinal sequences, images, audio recordings, etc. When the outcome is unstructured it is unclear how to assess RCT success and how to compute sample size. We show that kernel methods offer natural extensions to traditional biostatistics methods. We demonstrate our approach with the measurements of computer usage in a cohort of aging participants, some of which will…
In clinical trials, there is potential to improve precision and reduce the required sample size by appropriately adjusting for baseline variables in the statistical analysis. This is called covariate adjustment. Despite recommendations by regulatory agencies in favor of covariate adjustment, it remains underutilized leading to inefficient trials. We address two obstacles that make it challenging to use covariate adjustment. A first obstacle is the incompatibility of many covariate adjusted estimators with commonly used boundaries in group sequential designs (GSDs). A second obstacle is the unc…
A sequential multiple assignment randomized trial (SMART) facilitates comparison of multiple adaptive treatment strategies (ATSs) simultaneously. Previous studies have established a framework to test the homogeneity of multiple ATSs by a global Wald test through inverse probability weighting. SMARTs are generally lengthier than classical clinical trials due to the sequential nature of treatment randomization in multiple stages. Thus, it would be beneficial to add interim analyses allowing for early stop if overwhelming efficacy is observed. We introduce group sequential methods to SMARTs to fa…
Covariate adjustment is an important tool in the analysis of randomized clinical trials and observational studies. It can be used to increase efficiency and thus power, and to reduce possible bias. While most statistical tests in randomized clinical trials are nonparametric in nature, approaches for covariate adjustment typically rely on specific regression models, such as the linear model for a continuous outcome, the logistic regression model for a dichotomous outcome and the Cox model for survival time. Several recent efforts have focused on model-free covariate adjustment. This paper makes…
Clinical trials are complex and usually involve multiple objectives such as controlling type I error rate, increasing power to detect treatment difference, assigning more patients to better treatment, and more. In literature, both response-adaptive randomization (RAR) procedures (by changing randomization procedure sequentially) and sequential monitoring (by changing analysis procedure sequentially) have been proposed to achieve these objectives to some degree. In this paper, we propose to sequentially monitor response-adaptive randomized clinical trial and study it's properties. We prove that…
In randomized clinical trials, adjustments for baseline covariates at both design and analysis stages are highly encouraged by regulatory agencies. A recent trend is to use a model-assisted approach for covariate adjustment to gain credibility and efficiency while producing asymptotically valid inference even when the model is incorrect. In this article we present three considerations for better practice when model-assisted inference is applied to adjust for covariates under simple or covariate-adaptive randomized trials: (1) guaranteed efficiency gain: a model-assisted method should often gai…
We present a new estimator of the restricted mean survival time in randomized trials where there is right censoring that may depend on treatment and baseline variables. The proposed estimator leverages prognostic baseline variables to obtain equal or better asymptotic precision compared to traditional estimators. Under regularity conditions and random censoring within strata of treatment and baseline variables, the proposed estimator has the following features: (i) it is interpretable under violations of the proportional hazards assumption; (ii) it is consistent and at least as precise as the …
TechStep is a randomized trial of a mobile health interventions targeted towards transgender adolescents. The interventions include a short message system, a mobile-optimized web application, and electronic counseling. The primary outcomes are self-reported sexual risk behaviors and uptake of HIV preventing medication. In order that we may evaluate the efficacy of several different combinations of interventions, the trial has a sequentially randomized design. We use a causal framework to formalize the estimands of the primary and key secondary analyses of the TechStep trial data. Targeted mini…
Over the last decade, the number of randomized trials of programs to reduce intimate partner violence (IPV) has grown precipitously. However, most trials continue to measure and code violence using standards originally designed for global prevalence surveys. This choice may have consequences in terms of bias, power, and efficiency of trial estimates and may limit what we can learn about how programs are working. In this paper, we return to first principles to develop a generative model for violence reduction. We then use this model to better understand trade-offs in outcome coding choices via …
We examine study designs for extending (generalizing or transporting) causal inferences from a randomized trial to a target population. Specifically, we consider nested trial designs, where randomized individuals are nested within a sample from the target population, and non-nested trial designs, including composite dataset designs, where a randomized trial is combined with a separately obtained sample of non-randomized individuals from the target population. We show that the causal quantities that can be identified in each study design depend on what is known about the probability of sampling…
Clinical trials traditionally employ blinding as a design mechanism to reduce the influence of placebo effects. In practice, however, it can be difficult or impossible to blind study participants and unblinded trials are common in medical research. Here we show how instrumental variables can be used to quantify and disentangle treatment and placebo effects in randomized clinical trials comparing control and active treatments in the presence of confounders. The key idea is to use randomization to separately manipulate treatment assignment and psychological encouragement messages that increase t…
Extending (generalizing or transporting) causal inferences from a randomized trial to a target population requires ``generalizability'' or ``transportability'' assumptions, which state that randomized and non-randomized individuals are exchangeable conditional on baseline covariates. These assumptions are made on the basis of background knowledge, which is often uncertain or controversial, and need to be subjected to sensitivity analysis. We present simple methods for sensitivity analyses that do not require detailed background knowledge about specific unknown or unmeasured determinants of the…
When engagement with a randomized trial is driven by factors that affect the outcome or when trial engagement directly affects the outcome independent of treatment, the average treatment effect among trial participants is unlikely to generalize to a target population. In this paper, we use counterfactual and graphical causal models to examine under what conditions we can generalize causal inferences from a randomized trial to the target population of trial-eligible individuals. We offer an interpretation of generalizability analyses using the notion of a hypothetical intervention to "scale-up"…
The primary goal of randomized trials is to compare the effects of different interventions on some outcome of interest. In addition to the treatment assignment and outcome, data on baseline covariates, such as demographic characteristics or biomarker measurements, are typically collected. Incorporating such auxiliary covariates in the analysis of randomized trials can increase power, but questions remain about how to preserve type I error when incorporating such covariates in a flexible way, particularly when the number of randomized units is small. Using the Young Citizens study, a cluster-ra…
Background A key requirement for a useful power calculation is that the calculation mimic the data analysis that will be performed on the actual data, once it is observed. Close approximations may be difficult to achieve using analytic solutions, however, and thus Monte Carlo approaches, including both simulation and bootstrap resampling, are often attractive. One setting in which this is particularly true is cluster-randomized trial designs. However, Monte Carlo approaches are useful in many additional settings as well. Calculating power for cluster-randomized trials using analytic or simulat…
We often seek to estimate the causal effect of an exposure on a particular outcome in both randomized and observational settings. One such estimation method is the covariate-adjusted residuals estimator, which was designed for individually or cluster randomized trials. In this manuscript, we study the properties of this estimator and develop a new estimator that utilizes both covariate adjustment and inverse probability weighting We support our theoretical results with a simulation study and an application in an infectious disease setting. The covariate-adjusted residuals estimator is an effic…
We present methods for causally interpretable meta-analyses that combine information from multiple randomized trials to estimate potential (counterfactual) outcome means and average treatment effects in a target population. We consider identifiability conditions, derive implications of the conditions for the law of the observed data, and obtain identification results for transporting causal inferences from a collection of independent randomized trials to a new target population in which experimental data may not be available. We propose an estimator for the potential (counterfactual) outcome m…
The interAdapt R package is designed to be used by statisticians and clinical investigators to plan randomized trials. It can be used to determine if certain adaptive designs offer tangible benefits compared to standard designs, in the context of investigators' specific trial goals and constraints. Specifically, interAdapt compares the performance of trial designs with adaptive enrollment criteria versus standard (non-adaptive) group sequential trial designs. Performance is compared in terms of power, expected trial duration, and expected sample size. Users can either work directly in the R co…
To improve precision of estimation and power of testing hypothesis for an unconditional treatment effect in randomized clinical trials with binary outcomes, researchers and regulatory agencies recommend using g-computation as a reliable method of covariate adjustment. However, the practical application of g-computation is hindered by the lack of an explicit robust variance formula that can be used for different unconditional treatment effects of interest. To fill this gap, we provide explicit and robust variance estimators for g-computation estimators and demonstrate through simulations that t…
Randomized vaccine trials are used to assess vaccine efficacy and to characterize the durability of vaccine induced protection. If efficacy is demonstrated, the treatment of placebo volunteers becomes an issue. For COVID-19 vaccine trials, there is broad consensus that placebo volunteers should be offered a vaccine once efficacy has been established. This will likely lead to most placebo volunteers crossing over to the vaccine arm, thus complicating the assessment of long term durability. We show how to analyze durability following placebo crossover and demonstrate that the vaccine efficacy pr…
Simulation-Guided Planning of a Target Trial Emulated Cluster Randomized Trial for Mass Small-Quantity Lipid Nutrient Supplementation Combined with Expanded Program on Immunization in Rural Niger
We consider the problem of designing a prospective randomized trial in which the outcome data will be self-reported, and will involve sensitive topics. Our interest is in misreporting behavior, and how respondents' tendency to under- or overreport a binary outcome might affect the power of the experiment. We model the problem by assuming each individual in our study is a member of one "reporting class": a truth-teller, underreporter, overreporter, or false-teller. We show that the joint distribution of reporting classes and "response classes" (characterizing individuals' response to the treatm…
Randomized trials are considered the gold standard for estimating causal effects. Trial findings are often used to inform policy and programming efforts, yet their results may not generalize well to a relevant target population due to potential differences in effect moderators between the trial and population. Statistical methods have been developed to improve generalizability by combining trials and population data, and weighting the trial to resemble the population on baseline covariates.Large-scale surveys in fields such as health and education with complex survey designs are a logical sour…
Although there is much excitement surrounding the use of mobile and wearable technology for the purposes of delivering interventions as people go through their day-to-day lives, data analysis methods for constructing and optimizing digital interventions lag behind. Here, we elucidate data analysis methods for primary and secondary analyses of micro-randomized trials (MRTs), an experimental design to optimize digital just-in-time adaptive interventions. We provide a definition of causal "excursion" effects suitable for use in digital intervention development. We introduce the weighted and cente…
Considerations for the Integration of Randomized Controlled Trials and Real-World Data
University of California, Berkeley（加州大学伯克利分校）；Targeted ML Solutions, Inc.（Targeted ML Solutions 公司）；Adaptic Health；National Institute of Allergy and Infectious Diseases（美国国家过敏和传染病研究所）；Northeastern University（东北大学）；Brown University（布朗大学）；Novo Nordisk A/S（诺和诺德）
Background: Recent studies have shown a decrease in annualised relapse rates (ARRs) in placebo groups of randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS). Methods: We conducted a systematic literature search of RCTs in RMS. Data on eligibility criteria and baseline characteristics were extracted and tested for significant trends over time. A meta-regression was conducted to estimate their contribution to the decrease of trial ARRs over time. Results: We identified 56 studies. Patient age at baseline (p < 0.001), mean duration of multiple sclerosis (MS) at baseline (p =…
We propose a new integrated phase I/II trial design to identify the most efficacious dose combination that also satisfies certain safety requirements for drug-combination trials. We first take a Bayesian copula-type model for dose finding in phase I. After identifying a set of admissible doses, we immediately move the entire set forward to phase II. We propose a novel adaptive randomization scheme to favor assigning patients to more efficacious dose-combination arms. Our adaptive randomization scheme takes into account both the point estimate and variability of efficacy. By using a moving refe…
