Publication bias occurs when the publication of research results depends not only on the quality of the research but also on its nature and direction. The consequence is that published studies may not be truly representative of all valid studies undertaken, and this bias may threaten the validity of systematic reviews and meta-analyses - on which evidence-based medicine increasingly relies. Multivariate meta-analysis has recently received increasing attention for its ability reducing potential bias and improving statistical efficiency by borrowing information across outcomes. However, detecting and accounting for publication bias are more challenging in multivariate meta-analysis setting because some studies may be completely unpublished whereas some studies may selectively report part of multiple outcomes. In this paper, we propose a score test for jointly testing publication bias for multiple outcomes, which is novel to the multivariate setting. The proposed test is a natural multivariate extension of the univariate Egger's test, and can handle the above mentioned scenarios simultaneously, It accounts for correlations among multivariate outcomes, while allowing different types of outcomes, and can borrow information across outcomes. The proposed test is shown to be more powerful than the Egger's test, Begg's test and Trim and Fill method through simulation studies. Two data analyses are given to illustrate the performance of the proposed test in practice.
In meta-analyses, publication bias is a well-known, important and challenging issue because the validity of the results from a meta-analysis is threatened if the sample of studies retrieved for review is biased. One popular method to deal with publication bias is the Copas selection model, which provides a flexible sensitivity analysis for correcting the estimates with considerable insight into the data suppression mechanism. However, rigorous testing procedures under the Copas selection model to detect bias are lacking. To fill this gap, we develop a score-based test for detecting publication…
Prospective registration of study protocols in clinical trial registries is a useful way to minimize the risk of publication bias in meta-analysis, and several clinical trial registries are available nowadays. However, they are mainly used as a tool for searching studies and information submitted to the registries has not been utilized as efficiently as it could. In addressing publication bias in meta-analyses, sensitivity analysis with the Copas selection model is a more objective alternative to widely-used graphical methods such as the funnel-plot and the trim-and-fill method. Despite its ab…
The research on meta-analysis and particularly multivariate meta-analysis has been greatly influenced by the work of Ingram Olkin. This paper documents Olkin's contributions by way of citation counts and outlines several areas of contribution by Olkin and his academic descendants. An academic family tree is provided.
The present study investigates the performance of several statistical tests to detect publication bias in diagnostic meta-analysis by means of simulation. While bivariate models should be used to pool data from primary studies in diagnostic meta-analysis, univariate measures of diagnostic accuracy are preferable for the purpose of detecting publication bias. In contrast to earlier research, which focused solely on the diagnostic odds ratio or its logarithm ($\lnω$), the tests are combined with four different univariate measures of diagnostic accuracy. For each combination of test and univariat…
Systematic reviews aim to summarize all the available evidence relevant to a particular research question. If appropriate, the data from identified studies are quantitatively combined in a meta-analysis. Often only few studies regarding a particular research question exist. In these settings the estimation of the between-study heterogeneity is challenging. Furthermore, the assessment of publication bias is difficult as standard methods such as visual inspection or formal hypothesis tests in funnel plots do not provide adequate guidance. Previously, Henmi and Copas (Statistics in Medicine 2010,…
Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent betw…
Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models and they have some advantages over the widely used bias-correction method of the trim-and-fill method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals or require a complicated sensitivity analysis process. In this paper, we develop a simple publication bias adjustment method utilizing information on conducted but st…
Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection process…
The validity of conclusions from meta-analysis is potentially threatened by publication bias. Most existing procedures for correcting publication bias assume normality of the study-specific effects that account for between-study heterogeneity. However, this assumption may not be valid, and the performance of these bias correction procedures can be highly sensitive to departures from normality. Further, there exist few measures to quantify the magnitude of publication bias based on selection models. In this paper, we address both of these issues. First, we explore the use of heavy-tailed distri…
Multivariate meta-analysis of test accuracy studies when tests are evaluated in terms of sensitivity and specificity at more than one threshold represents an effective way to synthesize results by fully exploiting the data, if compared to univariate meta-analyses performed at each threshold independently. The approximation of logit transformations of sensitivities and specificities at different thresholds through a normal multivariate random-effects model is a recent proposal, that straightforwardly extends the bivariate models well recommended for the one threshold case. However, drawbacks of…
In meta-analysis of diagnostic test accuracy, summary receiver operating characteristic (SROC) is a recommended method to summarize the discriminant capacity of a diagnostic test in the presence of study-specific cutoff values and the area under the SROC (SAUC) gives the aggregate measure of test accuracy. SROC or SAUC can be estimated by bivariate modelling of pairs of sensitivity and specificity over the primary diagnostic studies. However, publication bias is a major threat to the validity of estimates in meta-analysis. To address this issue, we propose to adopt sensitivity analysis to make…
Multivariate meta-analysis can be adapted to a wide range of situations for multiple outcomes and multiple treatment groups when combining studies together. The within-study correlation between effect sizes is often assumed known in multivariate meta-analysis while it is not always known practically. In this paper, we propose a generic method to approximate the within-study covariance for effect sizes in multivariate meta-analysis and apply this method to the scenarios with multiple outcomes and one outcome with multiple treatment groups respectively.
Publication bias and p-hacking are two well-known phenomena that strongly affect the scientific literature and cause severe problems in meta-analyses. Due to these phenomena, the assumptions of meta-analyses are seriously violated and the results of the studies cannot be trusted. While publication bias is almost perfectly captured by the weighting function selection model, p-hacking is much harder to model and no definitive solution has been found yet. In this paper we propose to model both publication bias and p-hacking with selection models. We derive some properties for these models, and we…
Simulation studies are commonly used to evaluate the performance of newly developed meta-analysis methods. For methodology that is developed for an aggregated data meta-analysis, researchers often resort to simulation of the aggregated data directly, instead of simulating individual participant data from which the aggregated data would be calculated in reality. Clearly, distributional characteristics of the aggregated data statistics may be derived from distributional assumptions of the underlying individual data, but they are often not made explicit in publications. This paper provides the di…
Copas' method corrects a pooled estimate from an aggregated data meta-analysis for publication bias. Its performance has been studied for one particular mechanism of publication bias. We show through simulations that Copas' method is not robust against other realistic mechanisms. This questions the usefulness of Copas' method, since publication bias mechanisms are typically unknown in practice.
Studies often estimate associations between an outcome and multiple variates. For example, studies of diagnostic test accuracy estimate sensitivity and specificity, and studies of predictive and prognostic factors typically estimate associations for multiple factors. Meta-analysis is a family of statistical methods for synthesizing estimates across multiple studies. Multivariate models exist that account for within-study correlations and between-study heterogeneity. The number of parameters that must be estimated in existing models is quadratic in the number of variates (e.g., risk factors). T…
The purpose of the present paper is to assess the efficacy of confidence intervals for Rosenthal's fail-safe number. Although Rosenthal's estimator is highly used by researchers, its statistical properties are largely unexplored. First of all, we developed statistical theory which allowed us to produce confidence intervals for Rosenthal's fail-safe number.This was produced by discerning whether the number of studies analysed in a meta-analysis is fixed or random. Each case produces different variance estimators. For a given number of studies and a given distribution, we provided five variance …
Multivariate meta-analysis is gaining prominence in evidence synthesis research because it enables simultaneous synthesis of multiple correlated outcome data, and random-effects models have generally been used for addressing between-studies heterogeneities. However, coverage probabilities of confidence regions or intervals for standard inference methods for random-effects models (e.g., restricted maximum likelihood estimation) cannot retain their nominal confidence levels in general, especially when the number of synthesized studies is small because their validities depend on large sample appr…
Identifying compromise solutions across multiple treatment benefit-harm profiles in multivariate network meta-analysis
Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center-University of Freiburg（弗赖堡大学医学院医学生物计量与统计研究所）；Oslo Center for Biostatistics and Epidemiology, Department of Biostatistics, University of Oslo（奥斯陆大学生物统计系奥斯陆生物统计与流行病学中心）
Objective Bayesian inference procedures are derived for the parameters of the multivariate random effects model generalized to elliptically contoured distributions. The posterior for the overall mean vector and the between-study covariance matrix is deduced by assigning two noninformative priors to the model parameter, namely the Berger and Bernardo reference prior and the Jeffreys prior, whose analytical expressions are obtained under weak distributional assumptions. It is shown that the only condition needed for the posterior to be proper is that the sample size is larger than the dimension …
Adjusting for Outcome Reporting Bias in Meta-analysis: A Multiple Imputation Approach
Epidemiology, Biostatistics and Prevention Institute (EBPI)（流行病学、生物统计学与预防研究所）；Center for Reproducible Science and Research Synthesis (CRS)（可重复科学与研究综合中心）；University of Zurich (UZH)（苏黎世大学）
Scargle (2000) has discussed Rosenthal and Rubin's (1978) "fail-safe number" (FSN) method for estimating the number of unpublished studies in meta-analysis. He concluded that this FSN cannot possibly be correct because a central assumption the authors used conflicts with the very definition of publication bias. While this point has been made by others before (Elsahoff, 1978; Darlington, 1980; Thomas, 1985, Iyengar & Greenhouse, 1988), Scargle showed, by way of a simple 2-parameter model, how far off Rosenthal & Rubin' s estimate can be in practice. However, his results relied on the assumption…
