Sequential monitoring of randomized trials traditionally relies on parametric assumptions or asymptotic approximations. We discuss a family of nonparametric sequential tests - collectively called e-RT - for binary, event-only, and continuous endpoints. All active variants derive validity from the randomization mechanism. Using a betting framework, each test constructs a test martingale by sequentially wagering on randomized assignments or observed event labels before using the current label in the wealth update. Under the null hypothesis of no treatment effect, the expected wealth cannot grow, guaranteeing anytime-valid Type I error control regardless of stopping rule. The default e-RT posture is effect-size agnostic: monitoring can begin without specifying a hypothesized treatment effect. Alternatively, fixed design-calibrated wagers, including growth-rate-optimal (GROW) wagers, may be used as optional efficiency tools when a clinically meaningful design alternative is credible. We present simulation studies demonstrating calibration and power, and discuss the principled asymmetry in betting strategies across outcome types. These methods provide a conservative, assumption-light complement to model-based sequential analyses.
This paper proposes an adaptive randomization procedure for two-stage randomized controlled trials. The method uses data from a first-wave experiment in order to determine how to stratify in a second wave of the experiment, where the objective is to minimize the variance of an estimator for the average treatment effect (ATE). We consider selection from a class of stratified randomization procedures which we call stratification trees: these are procedures whose strata can be represented as decision trees, with differing treatment assignment probabilities across strata. By using the first wave t…
Randomization inference for stepped-wedge designs with noncompliance with application to a palliative care pragmatic trial
University of Chicago（芝加哥大学）；Data Science Institute, University of Chicago（芝加哥大学数据科学研究所）；University of Pennsylvania（宾夕法尼亚大学）；Department of Medicine, University of Pennsylvania（宾夕法尼亚大学医学系）；Department of Mathematics and Statistics, Mississippi State University（密西西比州立大学数学与统计系）；Department of Statistics and Data Science, University of Pennsylvania（宾夕法尼亚大学统计与数据科学系）；Yale School of Public Health（耶鲁公共卫生学院）
In comparative studies, such as in causal inference and clinical trials, balancing important covariates is often one of the most important concerns for both efficient and credible comparison. However, chance imbalance still exists in many randomized experiments. This phenomenon of covariate imbalance becomes much more serious as the number of covariates $p$ increases. To address this issue, we introduce a new randomization procedure, called pairwise sequential randomization (PSR). The proposed method allocates the units sequentially and adaptively, using information on the current level of imb…
Linear regression is arguably the most fundamental statistical model; however, the validity of its use in randomized clinical trials, despite being common practice, has never been crystal clear, particularly when stratified or covariate-adaptive randomization is used. In this paper, we investigate several of the most intuitive and commonly used regression models for estimating and inferring the treatment effect in randomized clinical trials. By allowing the regression model to be arbitrarily misspecified, we demonstrate that all these regression-based estimators robustly estimate the treatment…
In vaccine studies, investigators are often interested in studying effect modifiers of clinical treatment efficacy by biomarker-based principal strata, which is useful for selecting biomarker study endpoints for evaluating treatments in new trials, exploring biological mechanisms of clinical treatment efficacy, and studying mediators of clinical treatment efficacy. However, in trials where participants may enter the study with prior exposure therefore with variable baseline biomarker values, clinical treatment efficacy may depend jointly on a biomarker measured at baseline and measured at a fi…
In randomized controlled trials (RCTs), treatment is often assigned by stratified randomization. I show that among all stratified randomization schemes which treat all units with probability one half, a certain matched-pair design achieves the maximum statistical precision for estimating the average treatment effect (ATE). In an important special case, the optimal design pairs units according to the baseline outcome. In a simulation study based on datasets from 10 RCTs, this design lowers the standard error for the estimator of the ATE by 10% on average, and by up to 34%, relative to the origi…
Psychological interventions, especially those leveraging mobile and wireless technologies, often include multiple components that are delivered and adapted on multiple timescales (e.g., coaching sessions adapted monthly based on clinical progress, combined with motivational messages from a mobile device adapted daily based on the person's daily emotional state). The hybrid experimental design (HED) is a new experimental approach that enables researchers to answer scientific questions about the construction of psychological interventions in which components are delivered and adapted on differen…
Randomized experiments ensure robust causal inference that are critical to effective learning analytics research and practice. However, traditional randomized experiments, like A/B tests, are limiting in large scale digital learning environments. While traditional experiments can accurately compare two treatment options, they are less able to inform how to adapt interventions to continually meet learners' diverse needs. In this work, we introduce a trial design for developing adaptive interventions in scaled digital learning environments -- the sequential randomized trial (SRT). With the goal …
A small n, sequential, multiple assignment, randomized trial (snSMART) is a small sample, two-stage design where participants receive up to two treatments sequentially, but the second treatment depends on response to the first treatment. The treatment effect of interest in an snSMART is the first-stage response rate, but outcomes from both stages can be used to obtain more information from a small sample. A novel way to incorporate the outcomes from both stages applies power prior models, in which first stage outcomes from an snSMART are regarded as the primary data and second stage outcomes a…
In cluster randomized trials, patients are typically recruited after clusters are randomized, and the recruiters and patients may not be blinded to the assignment. This often leads to differential recruitment and consequently systematic differences in baseline characteristics of the recruited patients between intervention and control arms, inducing post-randomization selection bias. We rigorously define causal estimands in the presence of selection bias. We elucidate the conditions under which standard covariate adjustment methods can validly estimate these estimands. We further discuss the ad…
Covariate-adaptive randomization schemes such as the minimization and stratified permuted blocks are often applied in clinical trials to balance treatment assignments across prognostic factors. The existing theoretical developments on inference after covariate-adaptive randomization are mostly limited to situations where a correct model between the response and covariates can be specified or the randomization method has well-understood properties. Based on stratification with covariate levels utilized in randomization and a further adjusting for covariates not used in randomization, in this ar…
Due to the high cost and high failure rate of Phase III trials, seamless Phase II/III designs are more and more popular to trial efficiency. A potential attraction of Phase II/III design is to allow a randomized proof-of-concept stage prior to committing to the full cost of the Phase III trial. Population selection during the trial allows a trial to adapt and focus investment where it is most likely to provide patient benefit. Motivated by a clinical trial to find the population that potential benefits with dual-primary endpoints progression free survival (PFS) and overall survival (OS), we pr…
Randomized trials of infectious disease interventions, such as vaccines, often focus on groups of connected or potentially interacting individuals. When the pathogen of interest is transmissible between study subjects, interference may occur: individual infection outcomes may depend on treatments received by others. Epidemiologists have defined the primary causal effect of interest -- called the "susceptibility effect" -- as a contrast in infection risk under treatment versus no treatment, while holding exposure to infectiousness constant. A related quantity -- the "direct effect" -- is define…
In most clinical trials, patients are randomized with equal probability among treatments to obtain an unbiased estimate of the treatment effect. Response-adaptive randomization (RAR) has been proposed for ethical reasons, where the randomization ratio is tilted successively to favor the better performing treatment. However, the substantial disagreement regarding bias due to time-trends in adaptive randomization is not fully recognized. The type-I error is inflated in the traditional Bayesian RAR approaches when a time-trend is present. In our approach, patients are assigned in blocks and the r…
Randomization inference for treatment effects on survival outcomes
Wake Forest University（维克森林大学）；Wake Forest University School of Medicine（维克森林大学医学院）；University of Chicago（芝加哥大学）；The Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）
To maximize clinical benefit, clinicians routinely tailor treatment to the individual characteristics of each patient, where individualized treatment rules are needed and are of significant research interest to statisticians. In the covariate-adjusted randomization clinical trial with many covariates, we model the treatment effect with an unspecified function of a single index of the covariates and leave the baseline response completely arbitrary. We devise a class of estimators to consistently estimate the treatment effect function and its associated index while bypassing the estimation of th…
Longitudinal observational patient data can be used to investigate the causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for controlling for the time-dependent confounding that typically occurs. The most commonly used is inverse probability weighted estimation of marginal structural models (MSM-IPTW). An alternative, the sequential trials approach, is increasingly popular, in particular in combination with the target trial emulation framework. This approach involves creating a sequence of `trials' from new time origins, restricting to indi…
Technological advancements in the field of mobile devices and wearable sensors have helped overcome obstacles in the delivery of care, making it possible to deliver behavioral treatments anytime and anywhere. Increasingly the delivery of these treatments is triggered by predictions of risk or engagement which may have been impacted by prior treatments. Furthermore the treatments are often designed to have an impact on individuals over a span of time during which subsequent treatments may be provided. Here we discuss our work on the design of a mobile health smoking cessation experimental study…
Recently, there as been an increasing interest in the use of heavily restricted randomization designs which enforces balance on observed covariates in randomized controlled trials. However, when restrictions are strict, there is a risk that the treatment effect estimator will have a very high mean squared error. In this paper, we formalize this risk and propose a novel combinatoric-based approach to describe and address this issue. First, we validate our new approach by re-proving some known properties of complete randomization and restricted randomization. Second, we propose a novel diagnosti…
In medical research, a scenario often entertained is randomized controlled $2^2$ factorial design with a binary outcome. By utilizing the concept of potential outcomes, Dasgupta et al. (2015) proposed a randomization-based causal inference framework, allowing flexible and simultaneous estimations and inferences of the factorial effects. However, a fundamental challenge that Dasgupta et al. (2015)'s proposed methodology faces is that the sampling variance of the randomization-based factorial effect estimator is unidentifiable, rendering the corresponding classic "Neymanian" variance estimator s…
Clinical trials traditionally employ blinding as a design mechanism to reduce the influence of placebo effects. In practice, however, it can be difficult or impossible to blind study participants and unblinded trials are common in medical research. Here we show how instrumental variables can be used to quantify and disentangle treatment and placebo effects in randomized clinical trials comparing control and active treatments in the presence of confounders. The key idea is to use randomization to separately manipulate treatment assignment and psychological encouragement messages that increase t…
Customer scoring models are the core of scalable direct marketing. Uplift models provide an estimate of the incremental benefit from a treatment that is used for operational decision-making. Training and monitoring of uplift models require experimental data. However, the collection of data under randomized treatment assignment is costly, since random targeting deviates from an established targeting policy. To increase the cost-efficiency of experimentation and facilitate frequent data collection and model training, we introduce supervised randomization. It is a novel approach that integrates e…
Comparing causal estimands from sequential nested versus single point target trials: A simulation study
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Target RWE（目标真实世界证据）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）；Bloomberg School of Public Health, Johns Hopkins University（约翰霍普金斯大学彭博公共卫生学院）；Duke University（杜克大学）
Mobile health studies can leverage longitudinal sensor data from smartphones to guide the application of personalized medical interventions. In this paper, we propose that adoption of an instrumental variable approach for randomized trials with imperfect compliance provides a natural framework for personalized causal inference of medication response in mobile health studies. Randomized treatment suggestions can be easily delivered to the study participants via electronic messages popping up on the smart-phone screen. Under quite general assumptions we can identify the causal effect of the actu…
The statistical literature is known to be inconsistent in the use of the terms "permutation test" and "randomization test". Several authors succesfully argue that these terms should be used to refer to two distinct classes of tests and that there are major conceptual differences between these classes. The present paper explains an important difference in mathematical reasoning between these classes: a permutation test fundamentally requires that the set of permutations has a group structure, in the algebraic sense; the reasoning behind a randomization test is not based on such a group structur…
When a clinical trial is subject to a series of interim analyses as a result of which the study may be terminated or modified, final frequentist analyses need to take account of the design used. Failure to do so may result in overstated levels of significance, biased effect estimates and confidence intervals with inadequate coverage probabilities. A wide variety of valid methods of frequentist analysis have been devised for sequential designs comparing a single experimental treatment with a single control treatment. It is less clear how to perform the final analysis of a sequential or adaptive…
Response-Adaptive Randomization (RAR) is part of a wider class of data-dependent sampling algorithms, for which clinical trials are typically used as a motivating application. In that context, patient allocation to treatments is determined by randomization probabilities that change based on the accrued response data in order to achieve experimental goals. RAR has received abundant theoretical attention from the biostatistical literature since the 1930's and has been the subject of numerous debates. In the last decade, it has received renewed consideration from the applied and methodological co…
In randomized trials, appropriately adjusting for baseline variables and short-term outcomes can lead to increased precision and reduced sample size. We examine the impact of such adjustment in group sequential designs, i.e., designs with preplanned interim analyses where enrollment may be stopped early for efficacy or futility. We address the following questions: how much precision gain can be obtained by appropriately adjusting for baseline variables and a short-term outcome? How is this precision gain impacted by factors such as the proportion of pipeline participants (those who enrolled bu…
Interference occurs between individuals when the treatment (or exposure) of one individual affects the outcome of another individual. Previous work on causal inference methods in the presence of interference has focused on the setting where a priori it is assumed there is 'partial interference,' in the sense that individuals can be partitioned into groups wherein there is no interference between individuals in different groups. Bowers, Fredrickson, and Panagopoulos (2012) and Bowers, Fredrickson, and Aronow (2016) consider randomization-based inferential methods that allow for more general int…
We consider the problem of estimating and inferring treatment effects in randomized experiments. In practice, stratified randomization, or more generally, covariate-adaptive randomization, is routinely used in the design stage to balance the treatment allocations with respect to a few variables that are most relevant to the outcomes. Then, regression is performed in the analysis stage to adjust the remaining imbalances to yield more efficient treatment effect estimators. Building upon and unifying the recent results obtained for ordinary least squares adjusted estimators under covariate-adapti…
I develop a model of a randomized experiment with a binary intervention and a binary outcome. Potential outcomes in the intervention and control groups give rise to four types of participants. Fixing ideas such that the outcome is mortality, some participants would live regardless, others would be saved, others would be killed, and others would die regardless. These potential outcome types are not observable. However, I use the model to develop estimators of the number of participants of each type. The model relies on the randomization within the experiment and on deductive reasoning. I apply …
[Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions
In clinical trials, response-adaptive randomization (RAR) has the appealing ability to assign more subjects to better-performing treatments based on interim results. The traditional RAR strategy alters the randomization ratio on a patient-by-patient basis; this has been heavily criticized for bias due to time-trends. An alternate approach is blocked RAR, which groups patients together in blocks and recomputes the randomization ratio in a block-wise fashion; the final analysis is then stratified by block. However, the typical blocked RAR design divides patients into equal-sized blocks, which is…
