High-dimensional logistic regression is widely used in analyzing data with binary outcomes. In this paper, global testing and large-scale multiple testing for the regression coefficients are considered in both single- and two-regression settings. A test statistic for testing the global null hypothesis is constructed using a generalized low-dimensional projection for bias correction and its asymptotic null distribution is derived. A lower bound for the global testing is established, which shows that the proposed test is asymptotically minimax optimal over some sparsity range. For testing the individual coefficients simultaneously, multiple testing procedures are proposed and shown to control the false discovery rate (FDR) and falsely discovered variables (FDV) asymptotically. Simulation studies are carried out to examine the numerical performance of the proposed tests and their superiority over existing methods. The testing procedures are also illustrated by analyzing a data set of a metabolomics study that investigates the association between fecal metabolites and pediatric Crohn's disease and the effects of treatment on such associations.
In this paper a robust version of the classical Wald test statistics for linear hypothesis in the logistic regression model is introduced and its properties are explored. We study the problem under the assumption of random covariates although some ideas with non random covariates are also considered. The family of tests considered is based on the minimum density power divergence estimator instead of the maximum likelihood estimator and it is referred to as the Wald-type test statistic in the paper. We obtain the asymptotic distribution and also study the robustness properties of the Wald type …
Labeling patients in electronic health records with respect to their statuses of having a disease or condition, i.e. case or control statuses, has increasingly relied on prediction models using high-dimensional variables derived from structured and unstructured electronic health record data. A major hurdle currently is a lack of valid statistical inference methods for the case probability. In this paper, considering high-dimensional sparse logistic regression models for prediction, we propose a novel bias-corrected estimator for the case probability through the development of linearization and…
This article presents methods for the construction of two-sided and one-sided simultaneous hyperbolic bands for the logistic and probit regression models when the predictor variable is restricted to a given interval. The bands are constructed based on the asymptotic properties of the maximum likelihood estimators. Past articles have considered building two-sided asymptotic confidence bands for the logistic model, such as Piegorsch and Casella (1988). However, the confidence bands given by Piegorsch and Casella are conservative under a single interval restriction, and it is shown in this articl…
This paper studies whether Bayesian simultaneous three-way hypothesis tests can be logically coherent. Two types of results are obtained. First, under the standard error-wise constant loss, only for a limited set of models can a Bayes simultaneous test be logically coherent. Second, if more general loss functions are used, then it is possible to obtain Bayes simultaneous tests that are always logically coherent. An explicit example of such a loss function is provided.
Discovering important genes that account for the phenotype of interest has long been challenging in genomewide expression analysis. Analyses such as Gene Set Enrichment Analysis (GSEA) that incorporate pathway information have become widespread in hypothesis testing, but pathway-based approaches have been largely absent from regression methods due to the challenges of dealing with overlapping pathways and the resulting lack of available software. The R package grpreg is widely used to fit group lasso and other group-penalized regression models; in this study, we develop an extension, grpregOve…
In the last decade, the secondary use of large data from health systems, such as electronic health records, has demonstrated great promise in advancing biomedical discoveries and improving clinical decision making. However, there is an increasing concern about biases in association studies caused by misclassification in the binary outcomes derived from electronic health records. We revisit the classical logistic regression model with misclassified outcomes. Despite that local identification conditions in some related settings have been previously established, the global identification of such …
Negative control is a common technique in scientific investigations and broadly refers to the situation where a null effect (''negative result'') is expected. Motivated by a real proteomic dataset, we will present three promising and closely connected methods of using negative controls to assist simultaneous hypothesis testing. The first method uses negative controls to construct a permutation p-value for every hypothesis under investigation, and we give several sufficient conditions for such p-values to be valid and positive regression dependent on the set (PRDS) of true nulls. The second met…
Despite its common practice, statistical hypothesis testing presents challenges in interpretation. For instance, in the standard frequentist framework there is no control of the type II error. As a result, the non-rejection of the null hypothesis cannot reasonably be interpreted as its acceptance. We propose that this dilemma can be overcome by using agnostic hypothesis tests, since they can control the type I and II errors simultaneously. In order to make this idea operational, we show how to obtain agnostic hypothesis in typical models. For instance, we show how to build (unbiased) uniformly…
In the analysis of microarray data, and in some other contemporary statistical problems, it is not uncommon to apply hypothesis tests in a highly simultaneous way. The number, $ν$ say, of tests used can be much larger than the sample sizes, $n$, to which the tests are applied, yet we wish to calibrate the tests so that the overall level of the simultaneous test is accurate. Often the sampling distribution is quite different for each test, so there may not be an opportunity for combining data across samples. In this setting, how large can $ν$ be, as a function of $n$, before level accuracy beco…
For random field theory based multiple comparison corrections In brain imaging, it is often necessary to compute the distribution of the supremum of a random field. Unfortunately, computing the distribution of the supremum of the random field is not easy and requires satisfying many distributional assumptions that may not be true in real data. Thus, there is a need to come up with a different framework that does not use the traditional statistical hypothesis testing paradigm that requires to compute p-values. With this as a motivation, we can use a different approach called the logistic regres…
We develop the distribution of the number of hypotheses found to be statistically significant using the rule from Benjamini and Hochberg (1995) for controlling the false discovery rate (FDR). This distribution has both a small sample form and an asymptotic expression for testing many independent hypotheses simultaneously. We propose a parametric distribution $\,Ψ_I(\cdot)\,$ to approximate the marginal distribution of p-values under a non-uniform alternative hypothesis. This distribution is useful when there are many different alternative hypotheses and these are not individually well understo…
When dealing with the problem of simultaneously testing a large number of null hypotheses, a natural testing strategy is to first reduce the number of tested hypotheses by some selection (screening or filtering) process, and then to simultaneously test the selected hypotheses. The main advantage of this strategy is to greatly reduce the severe effect of high dimensions. However, the first screening or selection stage must be properly accounted for in order to maintain some type of error control. In this paper, we will introduce a selection rule based on a selection statistic that is independen…
In this article, we consider the problem of simultaneous testing of hypotheses when the individual test statistics are not necessarily independent. Specifically, we consider the problem of simultaneous testing of point null hypotheses against two-sided alternatives for the mean parameters of normally distributed random variables. We assume that conditionally given the vector of means, these random variables jointly follow a multivariate normal distribution with a known but arbitrary covariance matrix. We consider a Bayesian framework where each unknown mean parameter is modeled through a two-c…
Closed testing procedures are classically used for familywise error rate (FWER) control, but they can also be used to obtain simultaneous confidence bounds for the false discovery proportion (FDP) in all subsets of the hypotheses. In this paper we investigate the special case of closed testing with Simes local tests. We construct a novel fast and exact shortcut which we use to investigate the power of this method when the number of hypotheses goes to infinity. We show that, if a minimal amount of signal is present, the average power to detect false hypotheses at any desired FDP level does not …
We derive a computationally convenient formula for the large sample coverage probability of a confidence interval for a scalar parameter of interest following a preliminary hypothesis test that a specified vector parameter takes a given value in a general regression model. Previously, this large sample coverage probability could only be estimated by simulation. Our formula only requires the evaluation, by numerical integration, of either a double or triple integral, irrespective of the dimension of this specified vector parameter. We illustrate the application of this formula to a confidence i…
Importance sampling is a common technique for Monte Carlo approximation, including Monte Carlo approximation of p-values. Here it is shown that a simple correction of the usual importance sampling p-values creates valid p-values, meaning that a hypothesis test created by rejecting the null when the p-value is <= alpha will also have a type I error rate <= alpha. This correction uses the importance weight of the original observation, which gives valuable diagnostic information under the null hypothesis. Using the corrected p-values can be crucial for multiple testing and also in problems where …
Statistical dependence between hypotheses poses a significant challenge to the stability of large scale multiple hypotheses testing. Ignoring it often results in an unacceptably large spread in the false positive proportion even though the average value is acceptable [21, 39, 40, 49]. However, the statistical dependence structure of data is often unknown. Using a generic signalprocessing model, Bayesian multiple testing, and simulations, we demonstrate that the variance of the false positive proportion can be substantially reduced even under unknown short range dependence. We do this by modeli…
Logistic regression is used thousands of times a day to fit data, predict future outcomes, and assess the statistical significance of explanatory variables. When used for the purpose of statistical inference, logistic models produce p-values for the regression coefficients by using an approximation to the distribution of the likelihood-ratio test. Indeed, Wilks' theorem asserts that whenever we have a fixed number $p$ of variables, twice the log-likelihood ratio (LLR) $2Λ$ is distributed as a $χ^2_k$ variable in the limit of large sample sizes $n$; here, $k$ is the number of variables being te…
This article presents methods for constructing an asymptotic hyperbolic band under the multiple logistic regression model when the predictor variables are restricted to a specific region $\mathscr{X}$. Scheffé's method yields unnecessarily wide, and hence conservative, bands if the predictor variables can be restricted to a certain region. Piegorsch and Casella (1988) developed a procedure to build an asymptotic confidence band for the multiple logistic regression model over particular regions. Those regions are shown to be special cases of the region $\mathscr{X}$, which was first investigate…
Inference and prediction under the sparsity assumption have been a hot research topic in recent years. However, in practice, the sparsity assumption is difficult to test, and more importantly can usually be violated. In this paper, to study hypothesis test of any group of parameters under non-sparse high-dimensional linear models, we transform the null hypothesis to a testable moment condition and then use the self-normalization structure to construct moment test statistics under one-sample and two-sample cases, respectively. Compared to the one-sample case, the two-sample additionally require…
Many studies in biomedical and health sciences involve small sample sizes due to logistic or financial constraints. Often, identifying weak (but scientifically interesting) associations between a set of predictors and a response necessitates pooling datasets from multiple diverse labs or groups. While there is a rich literature in statistical machine learning to address distributional shifts and inference in multi-site datasets, it is less clear ${\it when}$ such pooling is guaranteed to help (and when it does not) -- independent of the inference algorithms we use. In this paper, we present a …
Modern statisticians are often presented with hundreds or thousands of hypothesis testing problems to evaluate at the same time, generated from new scientific technologies such as microarrays, medical and satellite imaging devices, or flow cytometry counters. The relevant statistical literature tends to begin with the tacit assumption that a single combined analysis, for instance, a False Discovery Rate assessment, should be applied to the entire set of problems at hand. This can be a dangerous assumption, as the examples in the paper show, leading to overly conservative or overly liberal conc…
The problem of multiple hypothesis testing arises when there are more than one hypothesis to be tested simultaneously for statistical significance. This is a very common situation in many data mining applications. For instance, assessing simultaneously the significance of all frequent itemsets of a single dataset entails a host of hypothesis, one for each itemset. A multiple hypothesis testing method is needed to control the number of false positives (Type I error). Our contribution in this paper is to extend the multiple hypothesis framework to be used with a generic data mining algorithm. We…
This paper infers a single parameter in non-sparse logistic regression models. By transforming the null hypothesis into a moment condition, we construct the test statistic and obtain the asymptotic null distribution. Numerical experiments show that our method performs well.
A multiple filter test (MFT) for the analysis and detection of rate change points in point processes on the line has been proposed recently. The underlying statistical test investigates the null hypothesis of constant rate. For that purpose, multiple filtered derivative processes are observed simultaneously. Under the null hypothesis, each process $G$ asymptotically takes the form \begin{align*} G \sim L, \end{align*} while $L$ is a zero-mean Gaussian process with unit variance. This result is used to derive a rejection threshold for statistical hypothesis testing. The purpose of this paper is…
Computational methods to simultaneously compare the predictive values of two diagnostic tests with missing data: EM-SEM algorithms and multiple imputation
Randomized response is one of the oldest and most well-known methods for analyzing confidential data. However, its utility for differentially private hypothesis testing is limited because it cannot achieve high privacy levels and low type I error rates simultaneously. In this article, we show how to overcome this issue with the subsample and aggregate technique. The result is a general-purpose method that can be used for both frequentist and Bayesian testing. {{We illustrate the performance of our proposal in three scenarios: goodness-of-fit testing for linear regression models, nonparametric …
Post-selection inference is a statistical technique for determining salient variables after model or variable selection. Recently, selective inference, a kind of post-selection inference framework, has garnered the attention in the statistics and machine learning communities. By conditioning on a specific variable selection procedure, selective inference can properly control for so-called selective type I error, which is a type I error conditional on a variable selection procedure, without imposing excessive additional computational costs. While selective inference can provide a valid hypothes…
The classical binary hypothesis testing problem is revisited. We notice that when one of the hypotheses is composite, there is an inherent difficulty in defining an optimality criterion that is both informative and well-justified. For testing in the simple normal location problem (that is, testing for the mean of multivariate Gaussians), we overcome the difficulty as follows. In this problem there exists a natural hardness order between parameters as for different parameters the error-probailities curves (when the parameter is known) are either identical, or one dominates the other. We can thu…
We aim to infer bioactivity of each chemical by assay endpoint combination, addressing sparsity of toxicology data. We propose a Bayesian hierarchical framework which borrows information across different chemicals and assay endpoints, facilitates out-of-sample prediction of activity for chemicals not yet assayed, quantifies uncertainty of predicted activity, and adjusts for multiplicity in hypothesis testing. Furthermore, this paper makes a novel attempt in toxicology to simultaneously model heteroscedastic errors and a nonparametric mean function, leading to a broader definition of activity w…
Suppose there are two unknown parameters, each parameter is the solution to an estimating equation, and the estimating equation of one parameter depends on the other parameter. The parameters can be jointly estimated by "stacking" their estimating equations and solving for both parameters simultaneously. Asymptotic confidence intervals are readily available for stacked estimating equations. We introduce a bootstrap-based hypothesis test for stacked estimating equations which does not rely on asymptotic approximations. Test statistics are constructed by splitting the sample in two, estimating t…
Genetic investigations often involve the testing of vast numbers of related hypotheses simultaneously. To control the overall error rate, a substantial penalty is required, making it difficult to detect signals of moderate strength. To improve the power in this setting, a number of authors have considered using weighted $p$-values, with the motivation often based upon the scientific plausibility of the hypotheses. We review this literature, derive optimal weights and show that the power is remarkably robust to misspecification of these weights. We consider two methods for choosing weights in p…
Large-sample properties of multiple imputation estimators for parameters of logistic regression with covariates missing at random separately or simultaneously
研究两组分类协变量分别或同时 MAR 时 logistic 回归 MI 估计量的渐近性质，提出修正方差估计，并通过模拟和逢甲夜市调查数据验证其优势。
Bayesian supervised predictive classifiers, hypothesis testing, and parametric estimation under Partition Exchangeability are implemented. The two classifiers presented are the marginal classifier (that assumes test data is i.i.d.) next to a more computationally costly but accurate simultaneous classifier (that finds a labelling for the entire test dataset at once based on simultanous use of all the test data to predict each label). We also provide the Maximum Likelihood Estimation (MLE) of the only underlying parameter of the partition exchangeability generative model as well as hypothesis te…
Detecting complex interactions among risk factors in case-control studies is a fundamental task in clinical and population research. However, though hypothesis testing using logistic regression (LR) is a convenient solution, the LR framework is poorly powered and ill-suited under several common circumstances in practice including missing or unmeasured risk factors, imperfectly correlated "surrogates", and multiple disease sub-types. The weakness of LR in these settings is related to the way in which the null hypothesis is defined. Here we propose the Asymmetric Independence Model (AIM) as a bi…
Sensor networks aim at monitoring their surroundings for event detection and object tracking. But due to failure or death of sensors, false signal can be transmitted. In this paper, we consider the problem of fault detection in wireless sensor network (WSN), in particular, addressing both the noise-related measurement error and sensor fault simultaneously in fault detection. We assume that the sensors are placed at the center of a square (or hexagonal) cell in region of interest (ROI) and, if the event occurs, it occurs at a particular cell of the ROI. We propose fault detection schemes that t…
School of Mathematical and Computer Sciences, Heriot-Watt University（赫瑞斯泰学院数学与计算机科学系，赫瑞瓦特大学）；School of Mathematics, University of Edinburgh（爱丁堡大学数学学院）；Maxwell Institute for Mathematical Sciences（麦克斯韦数学科学研究所）
Algorithms are now routinely used to make consequential decisions that affect human lives. Examples include college admissions, medical interventions or law enforcement. While algorithms empower us to harness all information hidden in vast amounts of data, they may inadvertently amplify existing biases in the available datasets. This concern has sparked increasing interest in fair machine learning, which aims to quantify and mitigate algorithmic discrimination. Indeed, machine learning models should undergo intensive tests to detect algorithmic biases before being deployed at scale. In this pa…
Due to the increasing availability of high-dimensional empirical applications in many research disciplines, valid simultaneous inference becomes more and more important. For instance, high-dimensional settings might arise in economic studies due to very rich data sets with many potential covariates or in the analysis of treatment heterogeneities. Also the evaluation of potentially more complicated (non-linear) functional forms of the regression relationship leads to many potential variables for which simultaneous inferential statements might be of interest. Here we provide a review of classica…
We propose a penalized likelihood method that simultaneously fits the multinomial logistic regression model and combines subsets of the response categories. The penalty is non differentiable when pairs of columns in the optimization variable are equal. This encourages pairwise equality of these columns in the estimator, which corresponds to response category combination. We use an alternating direction method of multipliers algorithm to compute the estimator and we discuss the algorithm's convergence. Prediction and model selection are also addressed.
