In a completely randomized experiment, the variances of treatment effect estimators in the finite population are usually not identifiable and hence not estimable. Although some estimable bounds of the variances have been established in the literature, few of them are derived in the presence of covariates.
In this paper, the difference-in-means estimator and the Wald estimator are considered in the completely randomized experiment with perfect compliance and noncompliance, respectively. Sharp bounds for the variances of these two estimators are established when covariates are available.
Furthermore, consistent estimators for such bounds are obtained, which can be used to shorten the confidence intervals and improve the power of tests. Confidence intervals are constructed based on the consistent estimators of the upper bounds, whose coverage rates are uniformly asymptotically guaranteed.
Simulations were conducted to evaluate the proposed methods. The proposed methods are also illustrated with two real data analyses.
To improve precision of estimation and power of testing hypothesis for an unconditional treatment effect in randomized clinical trials with binary outcomes, researchers and regulatory agencies recommend using g-computation as a reliable method of covariate adjustment. However, the practical application of g-computation is hindered by the lack of an explicit robust variance formula that can be used for different unconditional treatment effects of interest. To fill this gap, we provide explicit and robust variance estimators for g-computation estimators and demonstrate through simulations that t…
In observational studies, it is typically unrealistic to assume that treatments are randomly assigned, even conditional on adjusting for all observed covariates. Therefore, a sensitivity analysis is often needed to examine how hidden biases due to unobserved covariates would affect inferences on treatment effects. In matched observational studies where each treated unit is matched to one or multiple untreated controls for observed covariates, the Rosenbaum bounds sensitivity analysis is one of the most popular sensitivity analysis models. In this paper, we show that in the presence of interact…
We consider a longitudinal data structure consisting of baseline covariates, time-varying treatment variables, intermediate time-dependent covariates, and a possibly time dependent outcome. Previous studies have shown that estimating the variance of asymptotically linear estimators using empirical influence functions in this setting result in anti-conservative estimates with increasing magnitudes of positivity violations, leading to poor coverage and uncontrolled Type I errors. In this paper, we present two alternative approaches of estimating the variance of these estimators: (i) a robust app…
Covariate Balancing Value Estimation for Optimal Individualized Treatment Rules
School of Mathematical Sciences, Peking University（北京大学数学科学学院）；School of Mathematics and Statistics, Beijing Technology and Business University（北京工商大学数学与统计学院）
A result from a standard linear model course is that the variance of the ordinary least squares (OLS) coefficient of a variable will never decrease when including additional covariates. The variance inflation factor (VIF) measures the increase of the variance. Another result from a standard linear model or experimental design course is that including additional covariates in a linear model of the outcome on the treatment indicator will never increase the variance of the OLS coefficient of the treatment at least asymptotically. This technique is called the analysis of covariance (ANCOVA), which…
Linear regression is arguably the most fundamental statistical model; however, the validity of its use in randomized clinical trials, despite being common practice, has never been crystal clear, particularly when stratified or covariate-adaptive randomization is used. In this paper, we investigate several of the most intuitive and commonly used regression models for estimating and inferring the treatment effect in randomized clinical trials. By allowing the regression model to be arbitrarily misspecified, we demonstrate that all these regression-based estimators robustly estimate the treatment…
Propensity score methods have been shown to be powerful in obtaining efficient estimators of average treatment effect (ATE) from observational data, especially under the existence of confounding factors. When estimating, deciding which type of covariates need to be included in the propensity score function is important, since incorporating some unnecessary covariates may amplify both bias and variance of estimators of ATE. In this paper, we show that including additional instrumental variables that satisfy the exclusion restriction for outcome will do harm to the statistical efficiency. Also, …
We consider the problem of estimating and inferring treatment effects in randomized experiments. In practice, stratified randomization, or more generally, covariate-adaptive randomization, is routinely used in the design stage to balance the treatment allocations with respect to a few variables that are most relevant to the outcomes. Then, regression is performed in the analysis stage to adjust the remaining imbalances to yield more efficient treatment effect estimators. Building upon and unifying the recent results obtained for ordinary least squares adjusted estimators under covariate-adapti…
In clinical trials, the response of a given subject often depends on the selected treatment as well as on some covariates. We study optimal approximate designs of experiments in the models with treatment and covariate effects. We allow for the variances of the responses to depend on the chosen treatments, which introduces heteroscedasticity into the models. For estimating systems of treatment contrasts and linear functions of the covariates, we extend known results on D-optimality of product designs by providing product designs that are optimal with respect to general eigenvalue-based criteria…
Estimating causal effects from randomized experiments is central to clinical research. Reducing the statistical uncertainty in these analyses is an important objective for statisticians. Registries, prior trials, and health records constitute a growing compendium of historical data on patients under standard-of-care that may be exploitable to this end. However, most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control. Here, we propose a use of historical data that exploits linear covariate adjustment to improve the efficiency of trial…
Randomized controlled trials are not only the golden standard in medicine and vaccine trials but have spread to many other disciplines like behavioral economics, making it an important interdisciplinary tool for scientists. When designing randomized controlled trials, how to assign participants to treatments becomes a key issue. In particular in the presence of covariate factors, the assignment can significantly influence statistical properties and thereby the quality of the trial. Another key issue is the widely popular assumption among experimenters that participants do not influence each ot…
Covariate adjustment and methods of incorporating historical data in randomized clinical trials (RCTs) each provide opportunities to increase trial power. We unite these approaches for the analysis of RCTs with binary outcomes based on the Cochran-Mantel-Haenszel (CMH) test for marginal risk ratio (RR). In PROCOVA-CMH, subjects are stratified on a single prognostic covariate reflective of their predicted outcome on the control treatment (e.g. placebo). This prognostic score is generated based on baseline covariates through a model trained on historical data. We propose two closed-form prospect…
We provide a principled way for investigators to analyze randomized experiments when the number of covariates is large. Investigators often use linear multivariate regression to analyze randomized experiments instead of simply reporting the difference of means between treatment and control groups. Their aim is to reduce the variance of the estimated treatment effect by adjusting for covariates. If there are a large number of covariates relative to the number of observations, regression may perform poorly because of overfitting. In such cases, the Lasso may be helpful. We study the resulting La…
We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials.
We propose a method to reduce variance in treatment effect estimates in the setting of high-dimensional data. In particular, we introduce an approach for learning a metric to be used in matching treatment and control groups. The metric reduces variance in treatment effect estimates by weighting covariates related to the outcome and filtering out unrelated covariates.
We consider the problem of designing a randomized experiment on a source population to estimate the Average Treatment Effect (ATE) on a target population. We propose a novel approach which explicitly considers the target when designing the experiment on the source. Under the covariate shift assumption, we design an unbiased importance-weighted estimator for the target population's ATE. To reduce the variance of our estimator, we design a covariate balance condition (Target Balance) between the treatment and control groups based on the target population. We show that Target Balance achieves a h…
Estimation of a treatment effect by a regression discontinuity design faces a severe challenge when the running variable contains measurement errors since the errors smoothen the discontinuity on which the identification depends. The existing studies show that the variance of the measurement errors plays a vital role in both bias correction and identification under such situations. However, the methodologies to estimate the variance from data are relatively undeveloped. This paper proposes two estimators for the variance of measurement errors of running variables of sharp regression continuity…
Covariate adjustment is a commonly used method for total causal effect estimation. In recent years, graphical criteria have been developed to identify all valid adjustment sets, that is, all covariate sets that can be used for this purpose. Different valid adjustment sets typically provide total effect estimates of varying accuracies. Restricting ourselves to causal linear models, we introduce a graphical criterion to compare the asymptotic variances provided by certain valid adjustment sets. We employ this result to develop two further graphical tools. First, we introduce a simple variance re…
Covariate-adaptive randomization schemes such as the minimization and stratified permuted blocks are often applied in clinical trials to balance treatment assignments across prognostic factors. The existing theoretical developments on inference after covariate-adaptive randomization are mostly limited to situations where a correct model between the response and covariates can be specified or the randomization method has well-understood properties. Based on stratification with covariate levels utilized in randomization and a further adjusting for covariates not used in randomization, in this ar…
The ability to generalize experimental results from randomized control trials (RCTs) across locations is crucial for informing policy decisions in targeted regions. Such generalization is often hindered by the lack of identifiability due to unmeasured effect modifiers that compromise direct transport of treatment effect estimates from one location to another. We build upon sensitivity analysis in observational studies and propose an optimization procedure that allows us to get bounds on the treatment effects in targeted regions. Furthermore, we construct more informative bounds by balancing on…
In observational studies, propensity scores are commonly estimated by maxi- mum likelihood but may fail to balance high-dimensional pre-treatment covariates even after specification search. We introduce a general framework that unifies and generalizes several recent proposals to improve covariate balance when designing an observational study. In- stead of the likelihood function, we propose to optimize special loss functions---covariate balancing scoring rules (CBSR)---to estimate the propensity score. A CBSR is uniquely determined by the link function in the GLM and the estimand (a weighted a…
While attractive from a theoretical perspective, finely stratified experiments such as paired designs suffer from certain analytical limitations not present in block-randomized experiments with multiple treated and control individuals in each block. In short, when using an appropriately weighted difference-in-means to estimated the sample average treatment effect, the traditional variance estimator in a paired experiment is conservative unless the pairwise average treatment effects are constant across pairs; however, in more coarsely stratified experiments, the corresponding variance estimator…
Chance imbalance in baseline characteristics is common in randomized clinical trials. Regression adjustment such as the analysis of covariance (ANCOVA) is often used to account for imbalance and increase precision of the treatment effect estimate. An objective alternative is through inverse probability weighting (IPW) of the propensity scores. Although IPW and ANCOVA are asymptotically equivalent, the former may demonstrate inferior performance in finite samples. In this article, we point out that IPW is a special case of the general class of balancing weights, and advocate to use overlap weig…
In this paper, we study the estimation and inference of the quantile treatment effect under covariate-adaptive randomization. We propose two estimation methods: (1) the simple quantile regression and (2) the inverse propensity score weighted quantile regression. For the two estimators, we derive their asymptotic distributions uniformly over a compact set of quantile indexes, and show that, when the treatment assignment rule does not achieve strong balance, the inverse propensity score weighted estimator has a smaller asymptotic variance than the simple quantile regression estimator. For the in…
In sequential experiments, subjects become available for the study over a period of time, and covariates are often measured at the time of arrival. We consider the setting where the sample size is fixed but covariate values are unknown until subjects enrol. Given a model for the outcome, a sequential optimal design approach can be used to allocate treatments to minimize the variance of the treatment effect. We extend existing optimal design methodology so it can be used within a nonmyopic framework, where treatment allocation for the current subject depends not only on the treatments and covar…
Cluster-randomized trials (CRTs) involve randomizing entire groups of participants -- called clusters -- to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually-randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weig…
Covariate balance is crucial for unconfounded descriptive or causal comparisons. However, lack of balance is common in observational studies. This article considers weighting strategies for balancing covariates. We define a general class of weights---the balancing weights---that balance the weighted distributions of the covariates between treatment groups. These weights incorporate the propensity score to weight each group to an analyst-selected target population. This class unifies existing weighting methods, including commonly used weights such as inverse-probability weights as special cases…
In medical research, a scenario often entertained is randomized controlled $2^2$ factorial design with a binary outcome. By utilizing the concept of potential outcomes, Dasgupta et al. (2015) proposed a randomization-based causal inference framework, allowing flexible and simultaneous estimations and inferences of the factorial effects. However, a fundamental challenge that Dasgupta et al. (2015)'s proposed methodology faces is that the sampling variance of the randomization-based factorial effect estimator is unidentifiable, rendering the corresponding classic "Neymanian" variance estimator s…
