Power analyses are an important aspect of experimental design, because they help determine how experiments are implemented in practice. It is common to specify a desired level of power and compute the sample size necessary to obtain that power. Such calculations are well-known for completely randomized experiments, but there can be many benefits to using other experimental designs. For example, it has recently been established that rerandomization, where subjects are randomized until covariate balance is obtained, increases the precision of causal effect estimators. This work establishes the power of rerandomized treatment-control experiments, thereby allowing for sample size calculators. We find the surprising result that, while power is often greater under rerandomization than complete randomization, the opposite can occur for very small treatment effects. The reason is that inference under rerandomization can be relatively more conservative, in the sense that it can have a lower type-I error at the same nominal significance level, and this additional conservativeness adversely affects power. This surprising result is due to treatment effect heterogeneity, a quantity often ignored in power analyses. We find that heterogeneity increases power for large effect sizes but decreases power for small effect sizes.
We present an optimized rerandomization design procedure for a non-sequential treatment-control experiment. Randomized experiments are the gold standard for finding causal effects in nature. But sometimes random assignments result in unequal partitions of the treatment and control group visibly seen as imbalance in observed covariates. There can additionally be imbalance on unobserved covariates. Imbalance in either observed or unobserved covariates increases treatment effect estimator error inflating the width of confidence regions and reducing experimental power. "Rerandomization" is a strat…
We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation assumes a response that is linear in one observed covariate, an unobserved component and an additive treatment effect where the only randomness comes from the treatment allocations. It is well-known that the power depends on the allocations' imbalance in the observed covariate and this is the reason for the classic restricted designs such as rerandomization. We show that power is also affected by two oth…
Fisher's randomization test (FRT) delivers exact $p$-values under the strong null hypothesis of no treatment effect on any units whatsoever and allows for flexible covariate adjustment to improve the power. Of interest is whether the procedure could also be valid for testing the weak null hypothesis of zero average treatment effect. Towards this end, we evaluate two general strategies for FRT with covariate-adjusted test statistics: that based on the residuals from an outcome model with only the covariates, and that based on the output from an outcome model with both the treatment and the cova…
Trials enroll a large number of subjects in order to attain power, making them expensive and time-consuming. Sample size calculations are often performed with the assumption of an unadjusted analysis, even if the trial analysis plan specifies a more efficient estimator (e.g. ANCOVA). This leads to conservative estimates of required sample sizes and an opportunity for savings. Here we show that a relatively simple formula can be used to estimate the power of any two-arm, single-timepoint trial analyzed with a semiparametric efficient estimator, regardless of the domain of the outcome or kind of…
"Back-of-the-envelope" or "rule-of-thumb" calculations involving rough estimates of quantities play a central scientific role in developing intuition about the structure and behaviour of physical systems, for example in so-called `Fermi problems' in the physical sciences. Such calculations can be used to powerfully and quantitatively reason about biological systems, particularly at the interface between physics and biology. However, substantial uncertainties are often associated with values in cell biology, and performing calculations without taking this uncertainty into account may limit the …
Power and sample size calculations for causal mediation analysis with a binary mediator in randomized trials
Yau Mathematical Sciences Center, Tsinghua University（丘成桐数学科学中心，清华大学）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（北京国际数学研究中心）
A tutorial on conducting sample size and power calculations for detecting treatment effect heterogeneity in cluster randomized trials with linear mixed models
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Ottawa Hospital Research Institute（渥太华医院研究所）；University of Ottawa（渥太华大学）；University of Washington（华盛顿大学）；University of Pennsylvania（宾夕法尼亚大学）；Yale School of Medicine（耶鲁医学院）；Yale School of Public Health（耶鲁公共卫生学院）；Harvard Pilgrim Health Care Institute（哈佛皮尔格里姆健康护理研究所）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
Tang derived the exact power formulae for t tests and analysis of covariance (ANCOVA) in superiority, noninferiority and equivalence trials. The power calculation in equivalence trials can be simplified by using Owen's Q function, which is available in standard statistical software. We extend the exact power determination method for ANCOVA to unstratified and stratified multi-arm randomized trials. The method is applied to the design of multi-arm trials and gold standard noninferiority trials.
The reproducibility crisis has led to an increasing number of replication studies being conducted. Sample sizes for replication studies are often calculated using conditional power based on the effect estimate from the original study. However, this approach is not well suited as it ignores the uncertainty of the original result. Bayesian methods are used in clinical trials to incorporate prior information into power calculations. We propose to adapt this methodology to the replication framework and promote the use of predictive instead of conditional power in the design of replication studies.…
Background and objective: The stepped wedge cluster randomized trial is a study design increasingly used for public health intervention evaluations. Most previous literature focuses on power calculations for this particular type of cluster randomized trials for continuous outcomes, along with an approximation to this approach for binary outcomes. Although not accurate for binary outcomes, it has been widely used. To improve the approximation for binary outcomes, two new methods for stepped wedge designs (SWDs) of binary outcomes have recently been published. However, these new methods have not…
Comparison of Simulation-Guided Design to Closed-Form Power Calculations in Planning a Cluster Randomized Trial with Covariate-Constrained Randomization: A Case Study in Rural Chad
A Bayesian hierarchical push-forward method for cluster Randomized Controlled Trial power calculations in complex humanitarian settings
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Johns Hopkins Bloomberg School of Public Health（约翰霍普金斯大学布隆伯格公共卫生学院）；Community Partners International（社区合作伙伴国际）；The George Washington University（乔治华盛顿大学）
