We present a randomization-based inferential framework for experiments characterized by a strongly ignorable assignment mechanism where units have independent probabilities of receiving treatment. Previous works on randomization tests often assume these probabilities are equal within blocks of units. We consider the general case where they differ across units and show how to perform randomization tests and obtain point estimates and confidence intervals. Furthermore, we develop a rejection-sampling algorithm to conduct randomization-based inference conditional on ancillary statistics, covariate balance, or other statistics of interest. Through simulation we demonstrate how our algorithm can yield powerful randomization tests and thus precise inference. Our work also has implications for observational studies, which commonly assume a strongly ignorable assignment mechanism. Most methodologies for observational studies make additional modeling or asymptotic assumptions, while our framework only assumes the strongly ignorable assignment mechanism, and thus can be considered a minimal-assumption approach.
A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the meth…
Randomized block factorial experiments are widely used in industrial engineering, clinical trials, and social science. Researchers often use a linear model and analysis of covariance to analyze experimental results; however, limited studies have addressed the validity and robustness of the resulting inferences because assumptions for a linear model might not be justified by randomization in randomized block factorial experiments. In this paper, we establish a new finite population joint central limit theorem for usual (unadjusted) factorial effect estimators in randomized block $2^K$ factorial…
Recently Khmaladze has shown how to `rotate' one empirical process to another. This paper is the first to apply this transform when successive data points are generated by a single distributional family, but with covariates varying over the sample. The application is to Bernoulli trials, and new results show how group sizes rotated are related to the number of parameters, and explore the impact of different types of data generating processes. The utility of the rotation is clear: goodness of fit tests after rotation to a distribution free process are easily computed, show excellent convergence…
One-sided confidence intervals are presented for the average of non-identical Bernoulli parameters. These confidence intervals are expressed as analytical functions of the total number of Bernoulli games won, the number of rounds and the confidence level. Tightness of these bounds in the sense of Buehler, i.e. as the strictest possible monotonic intervals, is demonstrated for all confidence levels. A simple interval valid for all confidence levels is also provided with a tightness guarantee. Finally, an application of the proposed confidence intervals to sequential sampling is discussed.
In covariate-adaptive or response-adaptive randomization, the treatment assignment and outcome can be correlated. Under this situation, re-randomization tests are a straightforward and attractive method to provide valid statistical inference. In this paper, we investigate the number of repetitions in the re-randomization tests. This is motivated by the group sequential design in clinical trials, where the nominal significance bound can be very small at an interim analysis. Accordingly, re-randomization tests lead to a very large number of required repetitions, which may be computationally intr…
Concerns have been expressed over the validity of statistical inference under covariate-adaptive randomization despite the extensive use in clinical trials. In the literature, the inferential properties under covariate-adaptive randomization have been mainly studied for continuous responses; in particular, it is well known that the usual two sample t-test for treatment effect is typically conservative, in the sense that the actual test size is smaller than the nominal level. This phenomenon of invalid tests has also been found for generalized linear models without adjusting for the covariates …
In 2019, the World Health Organization identified dengue as one of the top ten global health threats. For the control of dengue, the Applying Wolbachia to Eliminate Dengue (AWED) study group conducted a cluster-randomized trial in Yogyakarta, Indonesia, and used a novel design, called the cluster-randomized test-negative design (CR-TND). This design can yield valid statistical inference with data collected by a passive surveillance system and thus has the advantage of cost-efficiency compared to traditional cluster-randomized trials. We investigate the statistical assumptions and properties of…
Given a composite null hypothesis H, test supermartingales are non-negative supermartingales with respect to H with initial value 1. Large values of test supermartingales provide evidence against H. As a result, test supermartingales are an effective tool for rejecting H, particularly when the p-values obtained are very small and serve as certificates against the null hypothesis. Examples include the rejection of local realism as an explanation of Bell test experiments in the foundations of physics and the certification of entanglement in quantum information science. Test supermartingales have…
Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization method to draw balanced assignments efficiently. We obtain t…
In clinical trials, response-adaptive randomization (RAR) has the appealing ability to assign more subjects to better-performing treatments based on interim results. The traditional RAR strategy alters the randomization ratio on a patient-by-patient basis; this has been heavily criticized for bias due to time-trends. An alternate approach is blocked RAR, which groups patients together in blocks and recomputes the randomization ratio in a block-wise fashion; the final analysis is then stratified by block. However, the typical blocked RAR design divides patients into equal-sized blocks, which is…
Two commonly used methods for improving precision and power in clinical trials are stratified randomization and covariate adjustment. However, many trials do not fully capitalize on the combined precision gains from these two methods, which can lead to wasted resources in terms of sample size and trial duration. We derive consistency and asymptotic normality of model-robust estimators that combine these two methods, and show that these estimators can lead to substantial gains in precision and power. Our theorems cover a class of estimators that handle continuous, binary, and time-to-event outc…
There is a movement in design of experiments away from the classic randomization put forward by Fisher, Cochran and others to one based on optimization. In fixed-sample trials comparing two groups, measurements of subjects are known in advance and subjects can be divided optimally into two groups based on a criterion of homogeneity or "imbalance" between the two groups. These designs are far from random. This paper seeks to understand the benefits and the costs over classic randomization in the context of different performance criterions such as Efron's worst-case analysis. In the criterion th…
Bayesian response adaptive clinical trials are currently evaluating experimental therapies for several diseases. Adaptive decisions, such as pre-planned variations of the randomization probabilities, attempt to accelerate the development of new treatments. The design of response adaptive trials, in most cases, requires time consuming simulation studies to describe operating characteristics, such as type I/II error rates, across plausible scenarios. We investigate large sample approximations of pivotal operating characteristics in Bayesian Uncertainty directed trial Designs (BUDs). A BUD trial …
Response adaptive randomization (RAR) is appealing from methodological, ethical, and pragmatic perspectives in the sense that subjects are more likely to be randomized to better performing treatment groups based on accumulating data. However, applications of RAR in confirmatory drug clinical trials with multiple active arms are limited largely due to its complexity, and lack of control of randomization ratios to different treatment groups. To address the aforementioned issues, we propose a Response Adaptive Block Randomization (RABR) design allowing arbitrarily pre-specified randomization rati…
Response-adaptive designs allow the randomization probabilities to change during the course of a trial based on cumulated response data, so that a greater proportion of patients can be allocated to the better performing treatments. A major concern over the use of response-adaptive designs in practice, particularly from a regulatory viewpoint, is controlling the type I error rate. In particular, we show that the naive z-test can have an inflated type I error rate even after applying a Bonferroni correction. Simulation studies have often been used to demonstrate error control, but do not provide…
In situations where it is difficult to enroll patients in randomized controlled trials, external data can improve efficiency and feasibility. In such cases, adaptive trial designs could be used to decrease enrollment in the control arm of the trial by updating the randomization ratio at the interim analysis. Updating the randomization ratio requires an estimate of the amount of information effectively borrowed from external data, which is typically done with a linear approximation. However, this linear approximation is not always a reliable estimate, which could potentially lead to sub-optimal…
Response-Adaptive Randomization (RAR) is part of a wider class of data-dependent sampling algorithms, for which clinical trials are typically used as a motivating application. In that context, patient allocation to treatments is determined by randomization probabilities that change based on the accrued response data in order to achieve experimental goals. RAR has received abundant theoretical attention from the biostatistical literature since the 1930's and has been the subject of numerous debates. In the last decade, it has received renewed consideration from the applied and methodological co…
There has been a split in the statistics community about the need for taking covariates into account in the design phase of a clinical trial. There are many advocates of using stratification and covariate-adaptive randomization to promote balance on certain known covariates. However, balance does not always promote efficiency or ensure more patients are assigned to the better treatment. We describe these procedures, including model-based procedures, for incorporating covariates into the design of clinical trials, and give examples where balance, efficiency and ethical considerations may be in …
Single subject or n-of-1 research designs have been widely used to evaluate treatment interventions. Many statistical procedures such as split-middle trend lines, regression trend line, Shewart-chart trend line, binomial tests, randomization tests and Tryon C-statistics have been used to analyze single-subject data, but they fail to control Type I error due to serially-dependent time-series observations. The interrupted time series analysis maintains Type I error but assumes that the intervention effect to be a linear trend change from baseline. In this paper, we consider an improved intervent…
We propose an optimal sequential methodology for obtaining confidence intervals for a binomial proportion $θ$. Assuming that an i.i.d. random sequence of Benoulli($θ$) trials is observed sequentially, we are interested in designing a)~a stopping time $T$ that will decide when is the best time to stop sampling the process, and b)~an optimum estimator $\hatθ_{T}$ that will provide the optimum center of the interval estimate of $θ$. We follow a semi-Bayesian approach, where we assume that there exists a prior distribution for $θ$, and our goal is to minimize the average number of samples while we…
This paper proposes an adaptive randomization procedure for two-stage randomized controlled trials. The method uses data from a first-wave experiment in order to determine how to stratify in a second wave of the experiment, where the objective is to minimize the variance of an estimator for the average treatment effect (ATE). We consider selection from a class of stratified randomization procedures which we call stratification trees: these are procedures whose strata can be represented as decision trees, with differing treatment assignment probabilities across strata. By using the first wave t…
[Invited Discussion] Randomization Tests to Address Disruptions in Clinical Trials: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions
Let $V(k)$ denote the waiting time, the number of trials needed to get a consecutive $k$ ones. We propose recurrence algorithms for the probability distribution function (pdf) and the probability generating function (pgf) of $V(k)$ in sequences of independent and Markov dependent Bernoulli trials using generalized Fibonacci sequences of order $k$. Maximum likelihood estimation (MLE) methods for the probability distributions are presented in both cases with simulation examples.
Linear regression is arguably the most fundamental statistical model; however, the validity of its use in randomized clinical trials, despite being common practice, has never been crystal clear, particularly when stratified or covariate-adaptive randomization is used. In this paper, we investigate several of the most intuitive and commonly used regression models for estimating and inferring the treatment effect in randomized clinical trials. By allowing the regression model to be arbitrarily misspecified, we demonstrate that all these regression-based estimators robustly estimate the treatment…
In randomized clinical trials, adjustments for baseline covariates at both design and analysis stages are highly encouraged by regulatory agencies. A recent trend is to use a model-assisted approach for covariate adjustment to gain credibility and efficiency while producing asymptotically valid inference even when the model is incorrect. In this article we present three considerations for better practice when model-assisted inference is applied to adjust for covariates under simple or covariate-adaptive randomized trials: (1) guaranteed efficiency gain: a model-assisted method should often gai…
In vaccine studies, investigators are often interested in studying effect modifiers of clinical treatment efficacy by biomarker-based principal strata, which is useful for selecting biomarker study endpoints for evaluating treatments in new trials, exploring biological mechanisms of clinical treatment efficacy, and studying mediators of clinical treatment efficacy. However, in trials where participants may enter the study with prior exposure therefore with variable baseline biomarker values, clinical treatment efficacy may depend jointly on a biomarker measured at baseline and measured at a fi…
In comparative studies, such as in causal inference and clinical trials, balancing important covariates is often one of the most important concerns for both efficient and credible comparison. However, chance imbalance still exists in many randomized experiments. This phenomenon of covariate imbalance becomes much more serious as the number of covariates $p$ increases. To address this issue, we introduce a new randomization procedure, called pairwise sequential randomization (PSR). The proposed method allocates the units sequentially and adaptively, using information on the current level of imb…
Covariate-adaptive randomization schemes such as the minimization and stratified permuted blocks are often applied in clinical trials to balance treatment assignments across prognostic factors. The existing theoretical developments on inference after covariate-adaptive randomization are mostly limited to situations where a correct model between the response and covariates can be specified or the randomization method has well-understood properties. Based on stratification with covariate levels utilized in randomization and a further adjusting for covariates not used in randomization, in this ar…
