The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While non-informative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.
Empirical prior distributions for treatment-by-subgroup interaction heterogeneity in random-effects meta-analysis
University Medical Center Göttingen（哥廷根大学医学中心）；DZHK (German Center for Cardiovascular Research)（德国心血管研究中心）；DZKJ (German Center for Child and Adolescent Health)（德国儿童与青少年健康中心）
Background: Pairwise and network meta-analyses using fixed effect and random effects models are commonly applied to synthesise evidence from randomised controlled trials. The models differ in their assumptions and the interpretation of the results. The model choice depends on the objective of the analysis and knowledge of the included studies. Fixed effect models are often used because there are too few studies with which to estimate the between-study standard deviation from the data alone. Objectives: The aim is to propose a framework for eliciting an informative prior distribution for the be…
We outline a Bayesian model-averaged meta-analysis for standardized mean differences in order to quantify evidence for both treatment effectiveness $δ$ and across-study heterogeneity $τ$. We construct four competing models by orthogonally combining two present-absent assumptions, one for the treatment effect and one for across-study heterogeneity. To inform the choice of prior distributions for the model parameters, we used 50% of the Cochrane Database of Systematic Reviews to specify rival prior distributions for $δ$ and $τ$. The relative predictive performance of the competing models and riv…
Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. We consider several likelihood-based inference methods. Confidence intervals are based on normal or Student-t approximations. We extract an empirical data set of 40 meta-analyses from recent reviews published by the German Institute for Quality and Efficiency in Health Care (IQWiG). Methods are then compared empirically as wel…
Objective: We investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding and between-trial heterogeneity. Study Design and Setting: Bayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias, compared to trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics. R…
Linear quantile regression models aim at providing a detailed and robust picture of the (conditional) response distribution as function of a set of observed covariates. Longitudinal data represent an interesting field of application of such models; due to their peculiar features, they represent a substantial challenge, in that the standard, cross-sectional, model representation needs to be extended for dealing with such kind of data. In fact, repeated observations from the same statistical unit poses a problem of dependence; in a conditional perspective, this dependence could be ascribed to so…
Genetic association analyses often involve data from multiple potentially-heterogeneous subgroups. The expected amount of heterogeneity can vary from modest (e.g., a typical meta-analysis) to large (e.g., a strong gene--environment interaction). However, existing statistical tools are limited in their ability to address such heterogeneity. Indeed, most genetic association meta-analyses use a "fixed effects" analysis, which assumes no heterogeneity. Here we develop and apply Bayesian association methods to address this problem. These methods are easy to apply (in the simplest case, requiring on…
There are many practical difficulties in the calibration of computer models to experimental data. One such complication is the fact that certain combinations of the calibration inputs can cause the code to output data lacking fundamental properties, or even to produce no output at all. In many cases the researchers want or need to exclude the possibility of these "failures" within their analyses. We propose a Bayesian (meta-)model in which the posterior distribution for the calibration parameters naturally excludes regions of the input space corresponding to failed runs. That is, we define a s…
Meta-regression is often used to form hypotheses about what is associated with heterogeneity in a meta-analysis and to estimate the extent to which effects can vary between cohorts and other distinguishing factors. However, study-level variables, called moderators, that are available and used in the meta-regression analysis will rarely explain all of the heterogeneity. Therefore, measuring and trying to understand residual heterogeneity is still important in a meta-regression, although it is not clear how some heterogeneity measures should be used in the meta-regression context. The coefficien…
In a bivariate meta-analysis the number of diagnostic studies involved is often very low so that frequentist methods may result in problems. Bayesian inference is attractive as informative priors that add small amount of information can stabilise the analysis without overwhelming the data. However, Bayesian analysis is often computationally demanding and the selection of the prior for the covariance matrix of the bivariate structure is crucial with little data. The integrated nested Laplace approximations (INLA) method provides an efficient solution to the computational issues by avoiding any …
In meta-analysis, the fraction of variance that is due to heterogeneity is known as I2. We show that the usual estimator of I2 is biased. The bias is largest when a meta-analysis has few studies and little heterogeneity. For example, with 7 studies and the true value of I2 at 0, the average estimate of I2 is .124. Estimates of I2 should be interpreted cautiously when the meta-analysis is small and the null hypothesis of homogeneity (I2=0) has not been rejected. In small meta-analyses, confidence intervals may be preferable to point estimates for I2.
Use of historical data in clinical trial design and analysis has shown various advantages such as reduction of within-study placebo-treated number of subjects and increase of study power. The meta-analytic-predictive (MAP) approach accounts with a hierarchical model for between-trial heterogeneity in order to derive an informative prior from historical (often control) data. In this paper, we introduce the package RBesT (R Bayesian Evidence Synthesis Tools) which implements the MAP approach with normal (known sampling standard deviation), binomial and Poisson endpoints. The hierarchical MAP mod…
Random-effects meta-analyses are used to combine evidence of treatment effects from multiple studies. Since treatment effects may vary across trials due to differences in study characteristics, heterogeneity in treatment effects between studies must be accounted for to achieve valid inference. The standard model for random-effects meta-analysis assumes approximately normal effect estimates and a normal random-effects model. However, standard methods based on this model ignore the uncertainty in estimating the between-trial heterogeneity. In the special setting of only two studies and in the pr…
Meta-analyses in orphan diseases and small populations generally face particular problems including small numbers of studies, small study sizes, and heterogeneity of results. However, the heterogeneity is difficult to estimate if only very few studies are included. Motivated by a systematic review in immunosuppression following liver transplantation in children we investigate the properties of a range of commonly used frequentist and Bayesian procedures in extensive simulation studies. Furthermore, the consequences for interval estimation of the common treatment effect in random effects meta-a…
Cochran's $Q$ statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used for estimation of between-study variance $τ^2$. Cochran's $Q$, or $Q_{IV}$, uses estimated inverse-variance weights which makes approximating its distribution rather complicated. As an alternative, we are investigating a new $Q$ statistic, $Q_F$, whose constant weights use only the studies' effective sample sizes. For standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of $Q_{IV}$ and $Q_F$, as the basis for tests of h…
A number of popular estimators of the between-study variance, $τ^2$, are based on the Cochran's $Q$ statistic for testing heterogeneity in meta analysis. We introduce new point and interval estimators of $τ^2$ for log-odds-ratio. These include new DerSimonian-Kacker-type moment estimators based on the first moment of $Q_F$, the $Q$ statistic with effective-sample-size weights, and novel median-unbiased estimators. We study, by simulation, bias and coverage of these new estimators of $τ^2$ and, for comparative purposes, bias and coverage of a number of well-known estimators based on the $Q$ sta…
An absolute measure of heterogeneity for lnOR in meta-analysis
Beijing University of Technology（北京工业大学）；Yunnan University of Finance and Economics（云南财经大学）；Renmin University of China（中国人民大学）；Hong Kong Baptist University（香港浸会大学）
Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct stan…
Meta-analysis combines pertinent information from existing studies to provide an overall estimate of population parameters/effect sizes, as well as to quantify and explain the differences between studies. However, testing the between-study heterogeneity is one of the most troublesome topics in meta-analysis research. Additionally, no methods have been proposed to test whether the size of the heterogeneity is larger than a specific level. The existing methods, such as the Q test and likelihood ratio (LR) tests, are criticized for their failure to control the Type I error rate and/or failure to …
Recent work has made important contributions in the development of causally-interpretable meta-analysis. These methods transport treatment effects estimated in a collection of randomized trials to a target population of interest. Ideally, estimates targeted toward a specific population are more interpretable and relevant to policy-makers and clinicians. However, between-study heterogeneity not arising from differences in the distribution of treatment effect modifiers can raise difficulties in synthesizing estimates across trials. The existence of such heterogeneity, including variations in tre…
Popular measures of meta-analysis heterogeneity, such as $I^2$, cannot be considered measures of population heterogeneity since they are dependant on samples sizes within studies. The coefficient of variation (CV) recently introduced and defined to be the heterogeneity variance divided by the absolute value of the overall mean effect does not suffer such shortcomings. However, very large CV values can occur when the effect is small making interpretation difficult. The purpose of this paper is two-fold. Firstly, we consider variants of the CV that exist in the interval (0, 1] making interpretat…
A variety of problems in random-effects meta-analysis arise from the conventional $Q$ statistic, which uses estimated inverse-variance (IV) weights. In previous work on standardized mean difference and log-odds-ratio, we found superior performance with an estimator of the overall effect whose weights use only group-level sample sizes. The $Q$ statistic with those weights has the form proposed by DerSimonian and Kacker. The distribution of this $Q$ and the $Q$ with IV weights must generally be approximated. We investigate approximations for those distributions, as a basis for testing and estima…
Simulations for estimation of heterogeneity variance and overall effect with constant and inverse-variance weights in meta-analysis of difference in standardized means (DSM)
Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent betw…
We consider estimation and inference in panel data models with additive unobserved individual specific heterogeneity in a high dimensional setting. The setting allows the number of time varying regressors to be larger than the sample size. To make informative estimation and inference feasible, we require that the overall contribution of the time varying variables after eliminating the individual specific heterogeneity can be captured by a relatively small number of the available variables whose identities are unknown. This restriction allows the problem of estimation to proceed as a variable s…
