Generalizing treatment effects from a randomized trial to a target population requires the assumption that potential outcome distributions are invariant across populations after conditioning on observed covariates. This assumption fails when unmeasured effect modifiers are distributed differently between trial participants and the target population. We develop a sensitivity analysis framework that bounds how much conclusions can change when this transportability assumption is violated. Our approach constrains the likelihood ratio between target and trial outcome densities by a scalar parameter $Λ\geq 1$, with $Λ= 1$ recovering standard transportability. For each $Λ$, we derive sharp bounds on the target average treatment effect -- the tightest interval guaranteed to contain the true effect under all data-generating processes compatible with the observed data and the sensitivity model. We show that the optimal likelihood ratios have a simple threshold structure, leading to a closed-form greedy algorithm that requires only sorting trial outcomes and redistributing probability mass. The resulting estimator runs in $O(n \log n)$ time and is consistent under standard regularity conditions. Simulations demonstrate that our bounds achieve nominal coverage when the true outcome shift falls within the specified $Λ$, provide substantially tighter intervals than worst-case bounds, and remain informative across a range of realistic violations of transportability.
Under what circumstances is it a threat to the parallel trends assumption required for Difference in Differences (DiD) studies if treatment decisions are based on past values of the outcome? We explore via simulation studies whether parallel trends holds across a grid of data generating processes generally conducive to parallel trends (random walk, Hidden Markov Model, and constant direct additive confounding), study designs (never treated, not yet treated, or later treated control groups), and outcome responsiveness of treatment (yes or no). We interpret the upshot of our simulation results t…
We propose a new, flexible model for inference of the effect of a binary treatment on a continuous outcome observed over subsequent time periods. The model allows to seperate association due to endogeneity of treatment selection from additional longitudinal association of the outcomes and hence unbiased estimation of dynamic treatment effects. We investigate the performance of the proposed method on simulated data and employ it to reanalyse data on the longitudinal effects of a long maternity leave on mothers' earnings after their return to the labour market.
Treatment effect estimation by comparing observed and predicted outcomes: conditions for valid inference and practical illustration
University Medical Center Utrecht, Utrecht University（乌得勒支大学医学中心，乌得勒支大学）；University Medical Center Groningen（格罗宁根大学医学中心）；MAASTRO Clinic（MAASTRO诊所）；Erasmus MC Cancer Institute（伊拉斯姆斯癌症研究所）
We consider a randomized controlled trial between two groups. The objective is to identify a population with characteristics such that the test therapy is more effective than the control therapy. Such a population is called a subgroup. This identification can be made by estimating the treatment effect and identifying interactions between treatments and covariates. To date, many methods have been proposed to identify subgroups for a single outcome. There are also multiple outcomes, but they are difficult to interpret and cannot be applied to outcomes other than continuous values. In this paper,…
In classical study designs, the aim is often to learn about the effects of a treatment or intervention on a single outcome; in many modern studies, however, data on multiple outcomes are collected and it is of interest to explore effects on multiple outcomes simultaneously. Such designs can be particularly useful in patient-centered research, where different outcomes might be more or less important to different patients. In this paper we propose scaled effect measures (via potential outcome notation) that translate effects on multiple outcomes to a common scale, using mean-variance and median-…
Assessing the causal effects of interventions on ordinal outcomes is an important objective of many educational and behavioral studies. Under the potential outcomes framework, we can define causal effects as comparisons between the potential outcomes under treatment and control. However, unfortunately, the average causal effect, often the parameter of interest, is difficult to interpret for ordinal outcomes. To address this challenge, we propose to use two causal parameters, which are defined as the probabilities that the treatment is beneficial and strictly beneficial for the experimental uni…
In randomized experiments, the actual treatments received by some experimental units may differ from their treatment assignments. This non-compliance issue often occurs in clinical trials, social experiments, and the applications of randomized experiments in many other fields. Under certain assumptions, the average treatment effect for the compliers is identifiable and equal to the ratio of the intention-to-treat effects of the potential outcomes to that of the potential treatment received. To improve the estimation efficiency, we propose three model-assisted estimators for the complier averag…
Comparative effectiveness research often involves evaluating the differences in the risks of an event of interest between two or more treatments using observational data. Often, the post-treatment outcome of interest is whether the event happens within a pre-specified time window, which leads to a binary outcome. One source of bias for estimating the causal treatment effect is the presence of confounders, which are usually controlled using propensity score-based methods. An additional source of bias is right-censoring, which occurs when the information on the outcome of interest is not complet…
The average treatment effect (ATE) is popularly used to assess the treatment effect. However, the ATE implicitly assumes a homogenous treatment effect even amongst individuals with different characteristics. In this paper, we mainly focus on assessing the treatment effect heterogeneity, which has important implications in designing the optimal individual treatment regimens and in policy making. The treatment benefit rate (TBR) and treatment harm rate (THR) have been defined to characterize the magnitude of heterogeneity for binary outcomes. When the outcomes are continuous, we extend the defin…
Semi-parametric methods are often used for the estimation of intervention effects on correlated outcomes in cluster-randomized trials (CRTs). When outcomes are missing at random (MAR), Inverse Probability Weighted (IPW) methods incorporating baseline covariates can be used to deal with informative missingness. Also, augmented generalized estimating equations (AUG) correct for imbalance in baseline covariates but need to be extended for MAR outcomes. However, in the presence of interactions between treatment and baseline covariates, neither method alone produces consistent estimates for the mar…
We are interested in the estimation of average treatment effects based on right-censored data of an observational study. We focus on causal inference of differences between t-year absolute event risks in a situation with competing risks. We derive doubly robust estimation equations and implement estimators for the nuisance parameters based on working regression models for the outcome, the censoring and the treatment distribution conditional on auxiliary baseline covariates. We use the functional delta method to show that our estimators are regular asymptotically linear estimators and estimate …
In medical research, a scenario often entertained is randomized controlled $2^2$ factorial design with a binary outcome. By utilizing the concept of potential outcomes, Dasgupta et al. (2015) proposed a randomization-based causal inference framework, allowing flexible and simultaneous estimations and inferences of the factorial effects. However, a fundamental challenge that Dasgupta et al. (2015)'s proposed methodology faces is that the sampling variance of the randomization-based factorial effect estimator is unidentifiable, rendering the corresponding classic "Neymanian" variance estimator s…
Estimation of Time-Varying Treatment Effects in a Joint Model for Longitudinal and Recurrent Event Outcomes in Mobile Health Data
University of Michigan（密歇根大学）；Institute for Social Research, University of Michigan（密歇根大学社会研究所）；Department of Population Health Sciences and Huntsman Cancer Insti（人口健康科学与亨斯曼癌症研究所）
Sibling fixed effects (FE) models are useful for estimating causal treatment effects while offsetting unobserved sibling-invariant confounding. However, treatment estimates are biased if an individual's outcome affects their sibling's outcome. We propose a robustness test for assessing the presence of outcome-to-outcome interference in linear two-sibling FE models. We regress a gain-score--the difference between siblings' continuous outcomes--on both siblings' treatments and on a pre-treatment observed FE. Under certain restrictions, the observed FE's partial regression coefficient signals the…
In the absence of a randomized experiment, a key assumption for drawing causal inference about treatment effects is the ignorable treatment assignment. Violations of the ignorability assumption may lead to biased treatment effect estimates. Sensitivity analysis helps gauge how causal conclusions will be altered in response to the potential magnitude of departure from the ignorability assumption. However, sensitivity analysis approaches for unmeasured confounding in the context of multiple treatments and binary outcomes are scarce. We propose a flexible Monte Carlo sensitivity analysis approach…
Cause-effect relationships are typically evaluated by comparing outcome responses to binary treatment values, representing two arms of a hypothetical randomized controlled trial. However, in certain applications, treatments of interest are continuous and multidimensional. For example, understanding the causal relationship between severity of radiation therapy, summarized by a multidimensional vector of radiation exposure values and post-treatment side effects is a problem of clinical interest in radiation oncology. An appropriate strategy for making interpretable causal conclusions is to reduc…
For ordinal outcomes, the average treatment effect is often ill-defined and hard to interpret. Echoing Agresti and Kateri (2017), we argue that the relative treatment effect can be a useful measure especially for ordinal outcomes, which is defined as $γ= \mathrm{pr}\{ Y_i(1) > Y_i(0) \} - \mathrm{pr}\{ Y_i(1) < Y_i(0) \}$, with $Y_i(1)$ and $Y_i(0)$ being the potential outcomes of unit $i$ under treatment and control, respectively. Given the marginal distributions of the potential outcomes, we derive the sharp bounds on $γ,$ which are identifiable parameters based on the observed data. Agresti…
In the presence of heterogeneity between the randomized controlled trial (RCT) participants and the target population, evaluating the treatment effect solely based on the RCT often leads to biased quantification of the real-world treatment effect. To address the problem of lack of generalizability for the treatment effect estimated by the RCT sample, we leverage observational studies with large samples that are representative of the target population. This paper concerns evaluating treatment effects on survival outcomes for a target population and considers a broad class of estimands that are …
We provide sufficient conditions for the identification of the heterogeneous treatment effects, defined as the conditional expectation for the differences of potential outcomes given the untreated outcome, under the nonignorable treatment condition and availability of the information on the marginal distribution of the untreated outcome. These functions are useful both to identify the average treatment effects (ATE) and to determine the treatment assignment policy. The identification holds in the following two general setups prevalent in applied studies: (i) a randomized controlled trial with …
To improve precision of estimation and power of testing hypothesis for an unconditional treatment effect in randomized clinical trials with binary outcomes, researchers and regulatory agencies recommend using g-computation as a reliable method of covariate adjustment. However, the practical application of g-computation is hindered by the lack of an explicit robust variance formula that can be used for different unconditional treatment effects of interest. To fill this gap, we provide explicit and robust variance estimators for g-computation estimators and demonstrate through simulations that t…
Causal mediation analysis aims at disentangling a treatment effect into an indirect mechanism operating through an intermediate outcome or mediator, as well as the direct effect of the treatment on the outcome of interest. However, the evaluation of direct and indirect effects is frequently complicated by non-ignorable selection into the treatment and/or mediator, even after controlling for observables, as well as sample selection/outcome attrition. We propose a method for bounding direct and indirect effects in the presence of such complications using a method that is based on a sequence of l…
We are interested in the distribution of treatment effects for an experiment where units are randomized to a treatment but outcomes are measured for pairs of units. For example, we might measure risk sharing links between households enrolled in a microfinance program, employment relationships between workers and firms exposed to a trade shock, or bids from bidders to items assigned to an auction format. Such a double randomized experimental design may be appropriate when there are social interactions, market externalities, or other spillovers across units assigned to the same treatment. Or it …
There is a growing literature on finding so-called optimal treatment rules, which are rules by which to assign treatment to individuals based on an individual's characteristics, such that a desired outcome is maximized. A related goal entails identifying individuals who are predicted to have a harmful indirect effect (the effect of treatment on an outcome through mediators) even in the presence of an overall beneficial effect of the treatment on the outcome. In some cases, the likelihood of a harmful indirect effect may outweigh a likely beneficial overall effect, and would be reason to cautio…
Many estimators of the average effect of a treatment on an outcome require estimation of the propensity score, the outcome regression, or both. It is often beneficial to utilize flexible techniques such as semiparametric regression or machine learning to estimate these quantities. However, optimal estimation of these regressions does not necessarily lead to optimal estimation of the average treatment effect, particularly in settings with strong instrumental variables. A recent proposal addressed these issues via the outcome-adaptive lasso, a penalized regression technique for estimating the pr…
Massachusetts Institute of Technology（麻省理工学院）；Facebook（脸书）
Learning heterogeneous treatment effects (HTEs) is an important problem across many fields. Most existing methods consider the setting with a single treatment arm and a single outcome metric. However, in many real world domains, experiments are run consistently - for example, in internet companies, A/B tests are run every day to measure the impacts of potential changes across many different metrics of interest. We show that even if an analyst cares only about the HTEs in one experiment for one metric, precision can be improved greatly by analyzing all of the data together to take advantage of …
Popular guidance on observational data analysis states that outcomes should be blinded when determining matching criteria or propensity scores. Such a blinding is informally said to maintain the "objectivity" of the analysis, and to prevent analysts from fishing for positive results by exploiting chance imbalances. Contrary to this notion, we show that outcome blinding is not a sufficient safeguard against fishing. Blinded and unblinded analysts can produce bias of the same order of magnitude in cases where the outcomes can be approximately predicted from baseline covariates. We illustrate thi…
This article proposes different tests for treatment effect heterogeneity when the outcome of interest, typically a duration variable, may be right-censored. The proposed tests study whether a policy 1) has zero distributional (average) effect for all subpopulations defined by covariate values, and 2) has homogeneous average effect across different subpopulations. The proposed tests are based on two-step Kaplan-Meier integrals and do not rely on parametric distributional assumptions, shape restrictions, or on restricting the potential treatment effect heterogeneity across different subpopulatio…
For ordinal outcomes, we construct sequences of alternative hypotheses in increasing departures from the sharp null hypothesis of zero treatment effect on each experimental unit, to help assess the powers of randomization tests in randomized treatment-control experiments.
