We develop sensitivity analyses for weak nulls in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast to randomized experiments and to paired observational studies, we show for general matched designs that over a large class of test statistics, any valid sensitivity analysis for the entirety of the weak null must be unnecessarily conservative if Fisher's sharp null of no treatment effect for any individual also holds. We present a sensitivity analysis valid for the weak null, and illustrate why it is generally conservative if the sharp null holds through new connections to inverse probability weighted estimators. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and that is valid for the weak null under additional restrictions which may be deemed benign by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under a host of reasonable data-generating processes. The procedures allow practitioners to assess robustness of estimated sample average treatment effects to hidden bias while allowing for unspecified effect heterogeneity in matched observational studies.
Investigating the causal relationship between exposure and the time-to-event outcome is an important topic in biomedical research. Previous literature has discussed the potential issues of using the hazard ratio as a marginal causal effect measure due to its noncollapsibility property. In this paper, we advocate using the restricted mean survival time (RMST) difference as the marginal causal effect measure, which is collapsible and has a simple interpretation as the difference of area under survival curves over a certain time horizon. To address both measured and unmeasured confounding, a matc…
Bias Mitigation in Matched Observational Studies with Continuous Treatments: Calipered Non-Bipartite Matching and Bias-Corrected Estimation and Inference
A distinctive feature of a clustered observational study is its multilevel or nested data structure arising from the assignment of treatment, in a non-random manner, to groups or clusters of units or individuals. Examples are ubiquitous in the health and social sciences including patients in hospitals, employees in firms, and students in schools. What is the optimal matching strategy in a clustered observational study? At first thought, one might start by matching clusters of individuals and then, within matched clusters, continue by matching individuals. But as we discuss in this paper, the o…
Matched observational studies are commonly used to study treatment effects in non-randomized data. After matching for observed confounders, there could remain bias from unobserved confounders. A standard way to address this problem is to do a sensitivity analysis. A sensitivity analysis asks how sensitive the result is to a hypothesized unmeasured confounder U. One method, known as simultaneous sensitivity analysis, has two sensitivity parameters: one relating U to treatment assignment and the other to response. This method assumes that in each matched set, U is distributed to make the bias wo…
The parametric g-formula is an approach to estimating causal effects of sustained treatment strategies from observational data. An often cited limitation of the parametric g-formula is the g-null paradox: a phenomenon in which model misspecification in the parametric g-formula is guaranteed under the conditions that motivate its use (i.e., when identifiability conditions hold and measured time-varying confounders are affected by past treatment). Many users of the parametric g-formula know they must acknowledge the g-null paradox as a limitation when reporting results but still require clarity …
In this paper, we present a new way of matching in observational studies that overcomes three limitations of existing matching approaches. First, it directly balances covariates with multi-valued treatments without requiring the generalized propensity score. Second, it builds self-weighted matched samples that are representative of a target population by design. Third, it can handle large data sets, with hundreds of thousands of observations, in a couple of minutes. The key insights of this new approach to matching are balancing the treatment groups relative to a target population and positing…
In observational studies, it is typically unrealistic to assume that treatments are randomly assigned, even conditional on adjusting for all observed covariates. Therefore, a sensitivity analysis is often needed to examine how hidden biases due to unobserved covariates would affect inferences on treatment effects. In matched observational studies where each treated unit is matched to one or multiple untreated controls for observed covariates, the Rosenbaum bounds sensitivity analysis is one of the most popular sensitivity analysis models. In this paper, we show that in the presence of interact…
Causal analyses for observational studies are often complicated by covariate imbalances among treatment groups, and matching methodologies alleviate this complication by finding subsets of treatment groups that exhibit covariate balance. It is widely agreed upon that covariate balance can serve as evidence that a matched dataset approximates a randomized experiment, but what kind of experiment does a matched dataset approximate? In this work, we develop a randomization test for the hypothesis that a matched dataset approximates a particular experimental design, such as complete randomization, …
In observational studies of treatment effects, matched samples are created so treated and control groups are similar in terms of observable covariates. Traditionally such matched samples consist of matched pairs. If a pair match fails to make treated and control units sufficiently comparable, alternative forms of matching may be necessary. One general strategy to improve balance is to match a variable number of control units to each treated unit. A more tailored strategy is to adopt a fine balance constraint. Under a fine balance constraint, a nominal covariate is exactly balanced, but it does…
In many scientific studies, it is of interest to determine whether an exposure has a causal effect on an outcome. In observational studies, this is a challenging task due to the presence of confounding variables that affect both the exposure and the outcome. Many methods have been developed to test for the presence of a causal effect when all such confounding variables are observed and when the exposure of interest is discrete. In this article, we propose a class of nonparametric tests of the null hypothesis that there is no average causal effect of an arbitrary univariate exposure on an outco…
An observational study may be biased for estimating causal effects by failing to control for unmeasured confounders. This paper proposes a new quantity called the "sensitivity value", which is defined as the minimum strength of unmeasured confounders needed to change the qualitative conclusions of a naive analysis assuming no unmeasured confounder. We establish the asymptotic normality of the sensitivity value in pair-matched observational studies. The theoretical results are then used to approximate the power of a sensitivity analysis and select the design of a study. We explore the potential…
Unquantified sources of uncertainty in observational causal analyses can break the integrity of the results. One would never want another analyst to repeat a calculation with the same dataset, using a seemingly identical procedure, only to find a different conclusion. However, as we show in this work, there is a typical source of uncertainty that is essentially never considered in observational causal studies: the choice of match assignment for matched groups, that is, which unit is matched to which other unit before a hypothesis test is conducted. The choice of match assignment is anything bu…
We propose new tests for assessing whether covariates in a treatment group and matched control group are balanced in observational studies. The tests exhibit high power under a wide range of multivariate alternatives, some of which existing tests have little power for. The asymptotic permutation null distributions of the proposed tests are studied and the p-values calculated through the asymptotic results work well in finite samples, facilitating the application of the test to large data sets. The tests are illustrated in a study of the effect of smoking on blood lead levels. The proposed test…
One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create high-quality matched samples, residual imbalance due to observed covariates not being well matched often persists. Although statistical tests have been developed to test the randomization assumption and its implications, few provide a means to quantify the level of residual confounding due to observed covariates not being well matched in matched samples. In this articl…
TCS Research（TCS研究院）；Indraprastha Institute of Information Technology Delhi（因德拉普拉斯信息技术学院德里分校）
Causal inference (CI) in observational studies has received a lot of attention in healthcare, education, ad attribution, policy evaluation, etc. Confounding is a typical hazard, where the context affects both, the treatment assignment and response. In a multiple treatment scenario, we propose the neural network based MultiMBNN, where we overcome confounding by employing generalized propensity score based matching, and learning balanced representations. We benchmark the performance on synthetic and real-world datasets using PEHE, and mean absolute percentage error over ATE as metrics. MultiMBNN…
Risk Set Matched Difference-in-Differences for the Analysis of Effect Modification in an Observational Study on the Impact of Gun Violence on Health Outcomes
提出结合profile matching和risk set matching的difference-in-differences框架研究异质性处理效应，发现非致命枪伤对幸存者及其家人健康有显著持续影响，且因受伤严重程度和意图而异。
We present a multivariate one-sided sensitivity analysis for matched observational studies, appropriate when the researcher has specified that a given causal mechanism should manifest itself in effects on multiple outcome variables in a known direction. The test statistic can be thought of as the solution to an adversarial game, where the researcher determines the best linear combination of test statistics to combat nature's presentation of the worst-case pattern of hidden bias. The corresponding optimization problem is convex, and can be solved efficiently even for reasonably sized observatio…
Manipulating a Continuous Instrumental Variable in an Observational Study of Premature Babies: Algorithm, Partial Identification Bounds, and Inference under Randomization and Biased Randomization Assumptions
Under the potential outcomes framework, causal effects are defined as comparisons between potential outcomes under treatment and control. To infer causal effects from randomized experiments, Neyman proposed to test the null hypothesis of zero average causal effect (Neyman's null), and Fisher proposed to test the null hypothesis of zero individual causal effect (Fisher's null). Although the subtle difference between Neyman's null and Fisher's null has caused lots of controversies and confusions for both theoretical and practical statisticians, a careful comparison between the two approaches has…
Multicategory Matched Learning for Estimating Optimal Individualized Treatment Rules in Observational Studies with Application to a Hepatocellular Carcinoma Study
Cardinality matching is a computational method for finding the largest possible number of matched pairs of exposed and unexposed individuals from an observational dataset, with specified patterns of baseline characteristics that represent a target population for analysis. This article explains the process of cardinality matching and how it simultaneously addresses the concerns of balance, sample size, and representativeness of matched samples in observational studies.
A sensitivity analysis in an observational study assesses the robustness of significant findings to unmeasured confounding. While sensitivity analyses in matched observational studies have been well addressed when there is a single outcome variable, accounting for multiple comparisons through the existing methods yields overly conservative results when there are multiple outcome variables of interest. This stems from the fact that unmeasured confounding cannot affect the probability of assignment to treatment differently depending on the outcome being analyzed. Existing methods allow this to o…
Propensity score matching (PSM) is the de-facto standard for estimating causal effects in observational studies. We show that PSM and its implementations are susceptible to several major drawbacks and illustrate these findings using a case study with $17,427$ patients. We derive four formal properties an optimal statistical matching algorithm should meet, and propose Deterministic Balancing Score exact Matching (DBSeM) which meets the aforementioned properties for an exact matching. Furthermore, we investigate one of the main problems of PSM, that is that common PSM results in one valid set of…
There is effect modification if the magnitude or stability of a treatment effect varies systematically with the level of an observed covariate. A larger or more stable treatment effect is typically less sensitive to bias from unmeasured covariates, so it is important to recognize effect modification when it is present. We illustrate a recent proposal for conducting a sensitivity analysis that empirically discovers effect modification by exploratory methods, but controls the family-wise error rate in discovered groups. The example concerns a study of mortality and use of the intensive care unit…
We examine over 40,000 International Protection (IP) determinations for non-EEA nationals covering a 16 year period in Ireland. We reconfigure these individual outcomes into a set of over 23,000 matched pairs based on combination of direct matching and propensity score matching. A key feature of this approach is that it replicates the statistical features of an experimental set-up where observational data only are to hand. As a consequence we are able to identify those explanatory factors that in fact contribute to the grant of IP. This is a key innovation in the analysis of protection outcome…
In an optimal nonbipartite match, a single population is divided into matched pairs to minimize a total distance within matched pairs. Nonbipartite matching has been used to strengthen instrumental variables in observational studies of treatment effects, essentially by forming pairs that are similar in terms of covariates but very different in the strength of encouragement to accept the treatment. Optimal nonbipartite matching is typically done using network optimization techniques that can be quick, running in polynomial time, but these techniques limit the tools available for matching. Inste…
Many recent efforts center on assessing the ability of real-world evidence (RWE) generated from non-randomized, observational data to produce results compatible with those from randomized controlled trials (RCTs). One noticeable endeavor is the RCT DUPLICATE initiative (Franklin et al., 2020, 2021). To better reconcile findings from an observational study and an RCT, or two observational studies based on different databases, it is desirable to eliminate differences between study populations. We outline an efficient, network-flow-based statistical matching algorithm that designs well-matched pa…
In matched observational studies where treatment assignment is not randomized, sensitivity analysis helps investigators determine how sensitive their estimated treatment effect is to some unmeasured con- founder. The standard approach calibrates the sensitivity analysis according to the worst case bias in a pair. This approach will result in a conservative sensitivity analysis if the worst case bias does not hold in every pair. In this paper, we show that for binary data, the standard approach can be calibrated in terms of the average bias in a pair rather than worst case bias. When the worst …
In an observational study, it is common to leverage known null effect to detect bias. One such strategy is to set aside a placebo sample -- a subset of data immune from the hypothesized cause-and-effect relationship. Existence of an effect in the placebo sample raises concern of unmeasured confounding bias while absence of it corroborates the causal conclusion. This paper establishes a formal framework for using a placebo sample to detect and remove bias. We state identification assumption, and develop estimation and inference methods based on outcome regression, inverse probability weighting,…
