Estimating treatment effects conditional on observed covariates can improve the ability to tailor treatments to particular individuals. Doing so effectively requires dealing with potential confounding, and also enough data to adequately estimate effect moderation. A recent influx of work has looked into estimating treatment effect heterogeneity using data from multiple randomized controlled trials and/or observational datasets. With many new methods available for assessing treatment effect heterogeneity using multiple studies, it is important to understand which methods are best used in which setting, how the methods compare to one another, and what needs to be done to continue progress in this field. This paper reviews these methods broken down by data setting: aggregate-level data, federated learning, and individual participant-level data. We define the conditional average treatment effect and discuss differences between parametric and nonparametric estimators, and we list key assumptions, both those that are required within a single study and those that are necessary for data combination. After describing existing approaches, we compare and contrast them and reveal open areas for future research. This review demonstrates that there are many possible approaches for estimating treatment effect heterogeneity through the combination of datasets, but that there is substantial work to be done to compare these methods through case studies and simulations, extend them to different settings, and refine them to account for various challenges present in real data.
In randomized trials, appropriately adjusting for baseline variables and short-term outcomes can lead to increased precision and reduced sample size. We examine the impact of such adjustment in group sequential designs, i.e., designs with preplanned interim analyses where enrollment may be stopped early for efficacy or futility. We address the following questions: how much precision gain can be obtained by appropriately adjusting for baseline variables and a short-term outcome? How is this precision gain impacted by factors such as the proportion of pipeline participants (those who enrolled bu…
Exploration of treatment effect heterogeneity (TEH) is an increasingly important aspect of modern statistical analysis for stratified medicine in randomised controlled trials (RCTs) as we start to gather more information on trial participants and wish to maximise the opportunities for learning from data. However, the analyst should refrain from including a large number of variables in a treatment interaction discovery stage. Because doing so can significantly dilute the power to detect any true outcome-predictive interactions between treatments and covariates. Current guidance is limited and m…
Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct stan…
Breakthroughs in cancer biology have defined new research programs emphasizing the development of therapies that target specific pathways in tumor cells. Innovations in clinical trial design have followed with master protocols defined by inclusive eligibility criteria and evaluations of multiple therapies and/or histologies. Consequently, characterization of subpopulation heterogeneity has become central to the formulation and selection of a study design. However, this transition to master protocols has led to challenges in identifying the optimal trial design and proper calibration of hyperpa…
One size fits all approaches to medicine have become a thing of the past as the understanding of individual differences grows. The paper introduces a test for the presence of heterogeneity in treatment effects in a clinical trial. Heterogeneity is assessed on the basis of the predicted individual treatment effects (PITE) framework and a permutation test is utilized to establish if significant heterogeneity is present. We first use the novel test to show that heterogeneity in the effects of interventions exists in the Amyotrophic Lateral Sclerosis Clinical Trials. We then show, using two differ…
Treatment effect heterogeneity occurs when individual characteristics influence the effect of a treatment. We propose a novel approach that combines prognostic score matching and conditional inference trees to characterize effect heterogeneity of a randomized binary treatment. One key feature that distinguishes our method from alternative approaches is that it controls the Type I error rate, i.e., the probability of identifying effect heterogeneity if none exists and retains the underlying subgroups. This feature makes our technique particularly appealing in the context of clinical trials, whe…
The primary benefit of identifying a valid surrogate marker is the ability to use it in a future trial to test for a treatment effect with shorter follow-up time or less cost. However, previous work has demonstrated potential heterogeneity in the utility of a surrogate marker. When such heterogeneity exists, existing methods that use the surrogate to test for a treatment effect while ignoring this heterogeneity may lead to inaccurate conclusions about the treatment effect, particularly when the patient population in the new study has a different mix of characteristics than the study used to ev…
Recent work has made important contributions in the development of causally-interpretable meta-analysis. These methods transport treatment effects estimated in a collection of randomized trials to a target population of interest. Ideally, estimates targeted toward a specific population are more interpretable and relevant to policy-makers and clinicians. However, between-study heterogeneity not arising from differences in the distribution of treatment effect modifiers can raise difficulties in synthesizing estimates across trials. The existence of such heterogeneity, including variations in tre…
The average treatment effect (ATE) is popularly used to assess the treatment effect. However, the ATE implicitly assumes a homogenous treatment effect even amongst individuals with different characteristics. In this paper, we mainly focus on assessing the treatment effect heterogeneity, which has important implications in designing the optimal individual treatment regimens and in policy making. The treatment benefit rate (TBR) and treatment harm rate (THR) have been defined to characterize the magnitude of heterogeneity for binary outcomes. When the outcomes are continuous, we extend the defin…
An important task in early phase drug development is to identify patients, which respond better or worse to an experimental treatment. While a variety of different subgroup identification methods have been developed for the situation of trials that study an experimental treatment and control, much less work has been done in the situation when patients are randomized to different dose groups. In this article we propose new strategies to perform subgroup analyses in dose-finding trials and discuss the challenges, which arise in this new setting. We consider model-based recursive partitioning, wh…
A tutorial on conducting sample size and power calculations for detecting treatment effect heterogeneity in cluster randomized trials with linear mixed models
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Ottawa Hospital Research Institute（渥太华医院研究所）；University of Ottawa（渥太华大学）；University of Washington（华盛顿大学）；University of Pennsylvania（宾夕法尼亚大学）；Yale School of Medicine（耶鲁医学院）；Yale School of Public Health（耶鲁公共卫生学院）；Harvard Pilgrim Health Care Institute（哈佛皮尔格里姆健康护理研究所）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
Meta-analyses in orphan diseases and small populations generally face particular problems including small numbers of studies, small study sizes, and heterogeneity of results. However, the heterogeneity is difficult to estimate if only very few studies are included. Motivated by a systematic review in immunosuppression following liver transplantation in children we investigate the properties of a range of commonly used frequentist and Bayesian procedures in extensive simulation studies. Furthermore, the consequences for interval estimation of the common treatment effect in random effects meta-a…
The micro-randomized trial (MRT) is a sequential randomized experimental design to empirically evaluate the effectiveness of mobile health (mHealth) intervention components that may be delivered at hundreds or thousands of decision points. MRTs have motivated a new class of causal estimands, termed "causal excursion effects", for which semiparametric inference can be conducted via a weighted, centered least squares criterion (Boruvka et al., 2018). Existing methods assume between-subject independence and non-interference. Deviations from these assumptions often occur. In this paper, causal exc…
Model-based recursive partitioning (MOB) can be used to identify subgroups with differing treatment effects. The detection rate of treatment-by-covariate interactions and the accuracy of identified subgroups using MOB depend strongly on the sample size. Using data from multiple randomized controlled clinical trials can overcome the problem of too small samples. However, naively pooling data from multiple trials may result in the identification of spurious subgroups as differences in study design, subject selection and other sources of between-trial heterogeneity are ignored. In order to accoun…
Many policy evaluations occur in settings where treatment is randomized at the cluster level, and there is treatment noncompliance within each cluster. For example, villages might be assigned to treatment and control, but residents in each village may choose to comply or not with their assigned treatment status. When noncompliance is present, the instrumental variables framework can be used to identify and estimate causal effects. While a large literature exists on instrumental variables estimation methods, relatively little work has been focused on settings with clustered treatments. Here, we…
We propose a test-based elastic integrative analysis of the randomized trial and real-world data to estimate treatment effect heterogeneity with a vector of known effect modifiers. When the real-world data are not subject to bias, our approach combines the trial and real-world data for efficient estimation. Utilizing the trial design, we construct a test to decide whether or not to use real-world data. We characterize the asymptotic distribution of the test-based estimator under local alternatives. We provide a data-adaptive procedure to select the test threshold that promises the smallest mea…
Targeted Data Fusion for Region-Specific Survival Effects in the AMP HIV Prevention Trials
North Carolina State University（北卡罗来纳州立大学）；Duke Clinical Research Institute（杜克临床研究所）；University of Pennsylvania（宾夕法尼亚大学）；Duke University School of Medicine（杜克大学医学院）；Fred Hutch Cancer Center（弗雷德·哈钦癌症中心）；Brown University（布朗大学）
Individualizing treatment assignment can improve outcomes for diseases with patient-to-patient variability in comparative treatment effects. When a clinical trial demonstrates that some patients improve on treatment while others do not, it is tempting to assume that treatment effect heterogeneity exists. However, if variability in response is mainly driven by factors other than treatment, investigating the extent to which covariate data can predict differential treatment response is a potential waste of resources. Motivated by recent meta-analyses assessing the potential of individualizing tre…
Chance imbalance in baseline characteristics is common in randomized clinical trials. Regression adjustment such as the analysis of covariance (ANCOVA) is often used to account for imbalance and increase precision of the treatment effect estimate. An objective alternative is through inverse probability weighting (IPW) of the propensity scores. Although IPW and ANCOVA are asymptotically equivalent, the former may demonstrate inferior performance in finite samples. In this article, we point out that IPW is a special case of the general class of balancing weights, and advocate to use overlap weig…
The participants in randomized trials and other studies used for causal inference are often not representative of the populations seen by clinical decision-makers. To account for differences between populations, researchers may consider standardizing results to a target population. We discuss several different types of homogeneity conditions that are relevant for standardization: Homogeneity of effect measures, homogeneity of counterfactual outcome state transition parameters, and homogeneity of counterfactual distributions. Each of these conditions can be used to show that a particular standa…
Randomized trials typically estimate average relative treatment effects, but decisions on the benefit of a treatment are possibly better informed by more individualized predictions of the absolute treatment effect. In case of a binary outcome, these predictions of absolute individualized treatment effect require knowledge of the individual's risk without treatment and incorporation of a possibly differential treatment effect (i.e. varying with patient characteristics). In this paper we lay out the causal structure of individualized treatment effect in terms of potential outcomes and describe t…
