In medical device comparison studies, equivalency test is commonly used to demonstrate two measurement methods agree up to a pre-specified performance goal based on the paired repeated measures. Such equivalency test often involves controlling the absolute differences that depend on both the mean and variance parameters, and poses some challenges for statistical analysis. For example, for the oximetry comparison study that motivates our research, FDA has clear guidelines approving an investigational pulse oximeter in comparison to a standard oximeter via testing the root mean squares (RMS), a composite measure of both mean and variance parameters. For the hypothesis testing of this composite measure, existing methods have been either exploratory or relying on the large-sample normal approximation with conservative and unsatisfactory performance. We develop a novel generalized pivotal test to rigorously and accurately test the system equivalency based on RMS. The proposed method has well-controlled type I error and favorable performance in our extensive numerical studies. When analyzing data from an oximetry comparison study, aiming to demonstrate performance equivalency between an FDA-cleared oximetry system and an investigational system, our proposed method resulted in a highly significant test result strongly supporting the system equivalency. We also provide efficient R programs for the proposed method in a publicly available R package. Considering that many practical equivalency studies of diagnostic devices are of small to medium sizes, our proposed method and software timely bridge an existing gap in the field.
Generalized linear models are widely used in many areas of knowledge. As in other classes of regression models, it is desirable to perform diagnostic analysis in generalized linear models using residuals that are approximately standard normally distributed. Diagnostic analysis in this class of models are usually performed using the standardized Pearson residual or the standardized deviance residual. The former has skewed distribution and the latter has negative mean, specially when the variance of the response variable is high. In this work, we introduce the adjusted quantile residual for gene…
Verification bias is a well known problem when the predictive ability of a diagnostic test has to be evaluated. In this paper, we discuss how to assess the accuracy of continuous-scale diagnostic tests in the presence of verification bias, when a three-class disease status is considered. In particular, we propose a fully nonparametric verification bias-corrected estimator of the ROC surface. Our approach is based on nearest-neighbor imputation and adopts generic smooth regression models for both the disease and the verification processes. Consistency and asymptotic normality of the proposed es…
Linear mixed models with large imbalanced crossed random effects structures pose severe computational problems for maximum likelihood estimation and for Bayesian analysis. The costs can grow as fast as $N^{3/2}$ when there are N observations. Such problems arise in any setting where the underlying factors satisfy a many to many relationship (instead of a nested one) and in electronic commerce applications, the N can be quite large. Methods that do not account for the correlation structure can greatly underestimate uncertainty. We propose a method of moments approach that takes account of the c…
We expect that some observers in perceptual signal detection experiments, such as radiologists, will make rational decisions, and therefore ratings from those observers are expected to form a convex ROC curve. However, measured and published curves are often not convex. This article examines the convexity-constrained nonparametric maximum likelihood estimator of the ROC curve given by Lloyd (2002). Like Lloyd we use the Pool Adjacent Violator Algorithm (PAVA) to construct the estimate of the convex curve. We present a direct proof that this estimate is a convex hull of the empirical ROC curve.…
The spatial dependence in mean has been well studied by plenty of models in a large strand of literature, however, the investigation of spatial dependence in variance is lagging significantly behind. The existing models for the spatial dependence in variance are scarce, with neither probabilistic structure nor statistical inference procedure being explored. To circumvent this deficiency, this paper proposes a new generalized logarithmic spatial heteroscedasticity model with exogenous variables (denoted by the log-SHE model) to study the spatial dependence in variance. For the log-SHE model, it…
The volume under the receiver operating characteristic surface (VUS) is useful for measuring the overall accuracy of a diagnostic test when the possible disease status belongs to one of three ordered categories. In medical studies, the VUS of a new test is typically estimated through a sample of measurements obtained by some suitable sample of patients. However, in many cases, only a subset of such patients has the true disease status assessed by a gold standard test. In this paper, for a continuous-scale diagnostic test, we propose four estimators of the VUS which accommodate for nonignorable…
Passing-Bablok regression is a standard tool for method and assay comparison studies thanks to its place in industry guidelines such as CLSI. Unfortunately, its computational cost is high as a naive approach requires O(n2) time. This makes it impossible to compute the Passing-Bablok regression estimator on large datasets. Additionally, even on smaller datasets it can be difficult to perform bootstrap-based inference. We introduce the first quasilinear time algorithm for the equivariant Passing-Bablok estimator. In contrast to the naive algorithm, our algorithm runs in O(n log(n)) expected time…
Combining matching and regression for causal inference provides double-robustness in removing treatment effect estimation bias due to confounding variables. In most real-world applications, however, treatment and control populations are not large enough for matching to achieve perfect or near-perfect balance on all confounding variables and their nonlinear/interaction functions, leading to trade-offs. [this fact is independent of regression, so a bit disjointed from first sentence.] Furthermore, variance is as important of a contributor as bias towards total error in small samples, and must th…
Given additional distributional information in the form of moment restrictions, kernel density and distribution function estimators with implied generalised empirical likelihood probabilities as weights achieve a reduction in variance due to the systematic use of this extra information. The particular interest here is the estimation of densities or distributions of (generalised) residuals in semi-parametric models defined by a finite number of moment restrictions. Such estimates are of great practical interest, being potentially of use for diagnostic purposes, including tests of parametric ass…
This paper analyzes the diagnostic of near multicollinearity in a multiple linear regression from auxiliary centered regressions (with intercept) and non-centered (without intercept). From these auxiliary regression, the centered and non-centered Variance Inflation Factors are calculated, respectively. It is also presented an expression that relate both of them.
In order to sample marginalized and/or hard-to-reach populations, respondent-driven sampling (RDS) and similar techniques reach their participants via peer referral. Under a Markov model for RDS, previous research has shown that if the typical participant refers too many contacts, then the variance of common estimators does not decay like $O(n^{-1})$, where $n$ is the sample size. This implies that confidence intervals will be far wider than under a typical sampling design. Here we show that generalized least squares (GLS) can effectively reduce the variance of RDS estimates. In particular, a …
In a bivariate meta-analysis the number of diagnostic studies involved is often very low so that frequentist methods may result in problems. Bayesian inference is attractive as informative priors that add small amount of information can stabilise the analysis without overwhelming the data. However, Bayesian analysis is often computationally demanding and the selection of the prior for the covariance matrix of the bivariate structure is crucial with little data. The integrated nested Laplace approximations (INLA) method provides an efficient solution to the computational issues by avoiding any …
This paper introduces the \proglang{R} package \pkg{meta4diag} for implementing Bayesian bivariate meta-analyses of diagnostic test studies. Our package \pkg{meta4diag} is a purpose-built front end of the \proglang{R} package \pkg{INLA}. While \pkg{INLA} offers full Bayesian inference for the large set of latent Gaussian models using integrated nested Laplace approximations, \pkg{meta4diag} extracts the features needed for bivariate meta-analysis and presents them in an intuitive way. It allows the user a straightforward model-specification and offers user-specific prior distributions. Further…
Meta-analyses of diagnostic test accuracy (DTA) studies have been gathering attention in research in clinical epidemiology and health technology development, and bivariate random-effects model is becoming a standard tool. However, standard inference methods usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations since they ignore the variability in the estimation of variance parameters. To overcome the difficulty, a new improved inference method, namely, an accurate confidence region for the meta-analysis of DTA, by asymptotically e…
Network meta-analysis (NMA) allow combining efficacy information from multiple comparisons from trials assessing different therapeutic interventions for a given disease and to estimate unobserved comparisons from a network of observed comparisons. Applying NMA on diagnostic accuracy studies is a statistical challenge given the inherent correlation of sensitivity and specificity. A conceptually simple and novel hierarchical arm-based (AB) model which expresses the logit transformed sensitivity and specificity as sum of fixed effects for test, correlated study-effects and a random error associat…
BART (Bayesian Additive Regression Trees) has become increasingly popular as a flexible and scalable nonparametric regression approach for modern applied statistics problems. For the practitioner dealing with large and complex nonlinear response surfaces, its advantages include a matrix-free formulation and the lack of a requirement to prespecify a confining regression basis. Although flexible in fitting the mean, BART has been limited by its reliance on a constant variance error model. This homoscedastic assumption is unrealistic in many applications. Alleviating this limitation, we propose H…
Estimating prevalence, the fraction of a population with a certain medical condition, is fundamental to epidemiology. Traditional methods rely on classification of test samples taken at random from a population. Such approaches to estimating prevalence are biased and have uncontrolled uncertainty. Here, we construct a new, unbiased, minimum variance estimator for prevalence. Recent result show that prevalence can be estimated from counting arguments that compare the fraction of samples in an arbitrary subset of the measurement space to what is expected from conditional probability models of th…
The current statistical procedures implemented in statistical software packages for pooling of diagnostic test accuracy data include hSROC regression and the bivariate random-effects meta-analysis model (BRMA). However, these models do not report the overall mean but rather the mean for a central study with random-effect equal to zero and have difficulties estimating the correlation between sensitivity and specificity when the number of studies in the meta-analysis is small and/or when the between-study variance is relatively large. This tutorial on advanced statistical methods for meta-analys…
A Causal Inference Approach for Evaluating Diagnostic Tests and AI-Enabled Medical Devices: From Effect Modification to Information-Augmented Decision-Making
Division of Biostatistics, University of California, Berkeley（加州大学伯克利分校生物统计系）
Recently, there as been an increasing interest in the use of heavily restricted randomization designs which enforces balance on observed covariates in randomized controlled trials. However, when restrictions are strict, there is a risk that the treatment effect estimator will have a very high mean squared error. In this paper, we formalize this risk and propose a novel combinatoric-based approach to describe and address this issue. First, we validate our new approach by re-proving some known properties of complete randomization and restricted randomization. Second, we propose a novel diagnosti…
