Simultaneous inference allows for the exploration of data while deciding on criteria for proclaiming discoveries. It was recently proved that all admissible post-hoc inference methods for true discoveries must employ closed testing. In this paper, we investigate efficient closed testing with local tests of a special form: thresholding a function of sums of test scores for the individual hypotheses. Under this special design, we propose a new statistic that quantifies the cost of multiplicity adjustments, and we develop fast (mostly linear-time) algorithms for post-hoc inference. Paired with recent advances in global null tests based on generalized means, our work instantiates a series of simultaneous inference methods that can handle many dependence structures and signal compositions. We provide guidance on the method choices via theoretical investigation of the conservativeness and sensitivity for different local tests, as well as simulations that find analogous behavior for local tests and full closed testing.
We provide some comments on the article `High-dimensional simultaneous inference with the bootstrap' by Ruben Dezeure, Peter Buhlmann and Cun-Hui Zhang.
We consider asymptotic distributions of maximum deviations of sample covariance matrices, a fundamental problem in high-dimensional inference of covariances. Under mild dependence conditions on the entries of the data matrices, we establish the Gumbel convergence of the maximum deviations. Our result substantially generalizes earlier ones where the entries are assumed to be independent and identically distributed, and it provides a theoretical foundation for high-dimensional simultaneous inference of covariances.
Let $X_1, \ldots, X_n\in\mathbb{R}^p$ be i.i.d. random vectors. We aim to perform simultaneous inference for the mean vector $\mathbb{E} (X_i)$ with finite polynomial moments and an ultra high dimension. Our approach is based on the truncated sample mean vector. A Gaussian approximation result is derived for the latter under the very mild finite polynomial ($(2+θ)$-th) moment condition and the dimension $p$ can be allowed to grow exponentially with the sample size $n$. Based on this result, we propose an innovative resampling method to construct simultaneous confidence intervals for mean vecto…
We propose new simultaneous inference methods for diagnostic trials with elaborate factorial designs. Instead of the commonly used total area under the receiver operating characteristic (ROC) curve, our parameters of interest are partial areas under ROC curve segments that represent clinically relevant biomarker cut-off values. We construct a nonparametric multiple contrast test for these parameters and show that it asymptotically controls the family-wise type one error rate. Finite sample properties of this test are investigated in a series of computer experiments. We provide empirical and th…
We study a broad class of asymmetric copulas introduced by Liebscher (2008) as a combination of multiple - usually symmetric - copulas. The main thrust of the paper is to provide new theoretical properties including exact tail dependence expressions and stability properties. A subclass of Liebscher copulas obtained by combining Fréchet copulas is studied in more details. We establish further dependence properties for copulas of this class and show that they are characterized by an arbitrary number of singular components. Furthermore, we introduce a novel iterative representation for general Li…
Simultaneous Inference for Covariance and Precision Matrices of Long-Range Dependent Time Series
Booth School of Business, The University of Chicago（芝加哥大学布斯商学院）；Marshall School of Business, University of Southern California（南加州大学马歇尔商学院）；Department of Statistics and Data Science, Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学统计与数据科学系）；Department of Statistics, The University of Chicago（芝加哥大学统计学系）
Some units from a population receive the same treatment that is different from treatments available for other reservoir populations. The minimal sufficient statistic $s$ for the pre-treatment $x$-covariates's distributions in the populations is the coarsest balancing score. $s$ is used to select matching units for simultaneous causal comparisons of multiple treatments.Necessary and sufficient conditions on the posterior distribution of the treatment variable (given $x$) determine whether a statistic is either sufficient or minimal sufficient for the x-covariates' distributions. Results in the …
The computational complexity of simultaneous inference methods in high-dimensional linear regression models quickly increases with the number variables. This paper proposes a computationally efficient method based on the Moore-Penrose pseudoinverse. Under a symmetry assumption on the available regressors, the estimators are normally distributed and accompanied by a closed-form expression for the standard errors that is free of tuning parameters. We study the numerical performance in Monte Carlo experiments that mimic the size of modern applications for which existing methods are computationall…
In this paper, we consider the time-inhomogeneous nonlinear time series regression for a general class of locally stationary time series. On one hand, we propose sieve nonparametric estimators for the time-varying regression functions which can achieve the min-max optimal rate. On the other hand, we develop a unified simultaneous inferential theory which can be used to conduct both structural and exact form testings on the functions. Our proposed statistics are powerful even under locally weak alternatives. We also propose a multiplier bootstrapping procedure for practical implementation. Our …
We propose a distributed bootstrap method for simultaneous inference on high-dimensional massive data that are stored and processed with many machines. The method produces an $\ell_\infty$-norm confidence region based on a communication-efficient de-biased lasso, and we propose an efficient cross-validation approach to tune the method at every iteration. We theoretically prove a lower bound on the number of communication rounds $τ_{\min}$ that warrants the statistical accuracy and efficiency. Furthermore, $τ_{\min}$ only increases logarithmically with the number of workers and the intrinsic di…
In various applications of regression analysis, in addition to errors in the dependent observations also errors in the predictor variables play a substantial role and need to be incorporated in the statistical modeling process. In this paper we consider a nonparametric measurement error model of Berkson type with fixed design regressors and centered random errors, which is in contrast to much existing work in which the predictors are taken as random observations with random noise. Based on an estimator that takes the error in the predictor into account and on a suitable Gaussian approximation,…
This paper proposes a bootstrap-assisted procedure to conduct simultaneous inference for high dimensional sparse linear models based on the recent de-sparsifying Lasso estimator (van de Geer et al. 2014). Our procedure allows the dimension of the parameter vector of interest to be exponentially larger than sample size, and it automatically accounts for the dependence within the de-sparsifying Lasso estimator. Moreover, our simultaneous testing method can be naturally coupled with the margin screening (Fan and Lv 2008) to enhance its power in sparse testing with a reduced computational cost, or…
A general class of time-varying regression models is considered in this paper. We estimate the regression coefficients by using local linear M-estimation. For these estimators, weak Bahadur representations are obtained and are used to construct simultaneous confidence bands. For practical implementation, we propose a bootstrap based method to circumvent the slow logarithmic convergence of the theoretical simultaneous bands. Our results substantially generalize and unify the treatments for several time-varying regression and auto-regression models. The performance for ARCH and GARCH models is s…
We describe semiparametric estimation and inference for causal effects using observational data from a single social network. Our asymptotic results are the first to allow for dependence of each observation on a growing number of other units as sample size increases. In addition, while previous methods have implicitly permitted only one of two possible sources of dependence among social network observations, we allow for both dependence due to transmission of information across network ties and for dependence due to latent similarities among nodes sharing ties. We propose new causal effects th…
We consider kernel estimation of marginal densities and regression functions of stationary processes. It is shown that for a wide class of time series, with proper centering and scaling, the maximum deviations of kernel density and regression estimates are asymptotically Gumbel. Our results substantially generalize earlier ones which were obtained under independence or beta mixing assumptions. The asymptotic results can be applied to assess patterns of marginal densities or regression functions via the construction of simultaneous confidence bands for which one can perform goodness-of-fit test…
We focus on the problem estimating a monotone trend function under additive and dependent noise. New point-wise confidence interval estimators under both short- and long-range dependent errors are introduced and studied. These intervals are obtained via the method of inversion of certain discrepancy statistics arising in hypothesis testing problems. The advantage of this approach is that it avoids the estimation of nuisance parameters such as the derivative of the unknown function, which existing methods are forced to deal with. While the methodology is motivated by earlier work in the indepen…
CarGurus（CarGurus公司）；Southwestern University of Finance and Economics（西南财经大学）；University of Pittsburgh（匹兹堡大学）；University of Michigan（密歇根大学）
Simultaneous inference after model selection is of critical importance to address scientific hypotheses involving a set of parameters. In this paper, we consider high-dimensional linear regression model in which a regularization procedure such as LASSO is applied to yield a sparse model. To establish a simultaneous post-model selection inference, we propose a method of contraction and expansion (MOCE) along the line of debiasing estimation that enables us to balance the bias-and-variance trade-off so that the super-sparsity assumption may be relaxed. We establish key theoretical results for th…
Inference and prediction under the sparsity assumption have been a hot research topic in recent years. However, in practice, the sparsity assumption is difficult to test, and more importantly can usually be violated. In this paper, to study hypothesis test of any group of parameters under non-sparse high-dimensional linear models, we transform the null hypothesis to a testable moment condition and then use the self-normalization structure to construct moment test statistics under one-sample and two-sample cases, respectively. Compared to the one-sample case, the two-sample additionally require…
We define the concept of dependence among multiple variables using maximum entropy techniques and introduce a graphical notation to denote the dependencies. Direct inference of information theoretic quantities from data uncovers dependencies even in undersampled regimes when the joint probability distribution cannot be reliably estimated. The method is tested on synthetic data. We anticipate it to be useful for inference of genetic circuits and other biological signaling networks.
Statistical analysis of multimodal imaging data is a challenging task, since the data involves high-dimensionality, strong spatial correlations and complex data structures. In this paper, we propose rigorous statistical testing procedures for making inferences on the complex dependence of multimodal imaging data. Motivated by the analysis of multi-task fMRI data in the Human Connectome Project (HCP) study, we particularly address three hypothesis testing problems: (a) testing independence among imaging modalities over brain regions, (b) testing independence between brain regions within imaging…
This paper addresses inference in large panel data models in the presence of both cross-sectional and temporal dependence of unknown form. We are interested in making inferences that do not rely on the choice of any smoothing parameter as is the case with the often employed "HAC" estimator for the covariance matrix. To that end, we propose a cluster estimator for the asymptotic covariance of the estimators and valid bootstrap schemes that do not require the selection of a bandwidth or smoothing parameter and accommodate the nonparametric nature of both temporal and cross-sectional dependence. …
Strict stationarity is a common assumption used in the time series literature in order to derive asymptotic distributional results for second-order statistics, like sample autocovariances and sample autocorrelations. Focusing on weak stationarity, this paper derives the asymptotic distribution of the maximum of sample autocovariances and sample autocorrelations under weak conditions by using Gaussian approximation techniques. The asymptotic theory for parameter estimation obtained by fitting a (linear) autoregressive model to a general weakly stationary time series is revisited and a Gaussian …
We show that a dynamic logit model for binary panel data allowing for state dependence and unobserved heterogeneity may be accurately approximated by a quadratic exponential model, the parameters of which have the same interpretation that they have in the true model. We also show how we can eliminate the parameters for the unobserved heterogeneity from the approximating model by conditioning on the total scores, i.e. sum of the response variables for any individual in the panel. This allows to construct an approximate conditional likelihood for the dynamic logit model, by maximizing which we c…
We consider the least-squares regression problem with unknown noise variance, where the observed data points are allowed to be corrupted by outliers. Building on the median-of-means (MOM) method introduced by Lecue and Lerasle Ann.Statist.48(2):906-931(April 2020) in the case of known noise variance, we propose a general MOM approach for simultaneous inference of both the regression function and the noise variance, requiring only an upper bound on the noise level. Interestingly, this generalization requires care due to regularity issues that are intrinsic to the underlying convex-concave optim…
In this article, we consider the problem of simultaneous testing of hypotheses when the individual test statistics are not necessarily independent. Specifically, we consider the problem of simultaneous testing of point null hypotheses against two-sided alternatives for the mean parameters of normally distributed random variables. We assume that conditionally given the vector of means, these random variables jointly follow a multivariate normal distribution with a known but arbitrary covariance matrix. We consider a Bayesian framework where each unknown mean parameter is modeled through a two-c…
This paper discusses the statistical inference problem associated with testing for dependence between two continuous random variables using Kendall's $τ$ in the context of the missing data problem. We prove the worst-case identified set for this measure of association always includes zero. The consequence of this result is that robust inference for dependence using Kendall's $τ$, where robustness is with respect to the form of the missingness-generating process, is impossible.
A variety of methods have been proposed for inference about extreme dependence for multivariate or spatially-indexed stochastic processes and time series. Most of these proceed by first transforming data to some specific extreme value marginal distribution, often the unit Fréchet, then fitting a family of max-stable processes to the transformed data and exploring dependence within the framework of that model. The marginal transformation, model selection, and model fitting are all possible sources of misspecification in this approach. We propose an alternative model-free approach, based on the …
We consider the problem of inference after model selection under weak assumptions in the time series setting. Even when the data are not independent, we show that sample splitting remains asymptotically valid as long as the process satisfies appropriate weak dependence conditions and the functional of interest is suitably well-behaved. In addition, if the inference targets are appropriately defined, we demonstrate that valid statistical inference is possible without assuming stationarity. As a working example, we consider post-selection inference for regression coefficients under a random desi…
We consider the problem of constructing confidence intervals for the locations of change points in a high-dimensional mean shift model. To that end, we develop a locally refitted least squares estimator and obtain component-wise and simultaneous rates of estimation of the underlying change points. The simultaneous rate is the sharpest available in the literature by at least a factor of $\log p,$ while the component-wise one is optimal. These results enable existence of limiting distributions. Component-wise distributions are characterized under both vanishing and non-vanishing jump size regime…
It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid ``post-selection inference'' by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all …
Online simultaneous inference for quantiles via smoothed stochastic gradient descent
Department of Mathematics and Statistics（数学与统计学系）；Washington University in St.Louis（圣路易斯华盛顿大学）；Chair of Statistics（统计学教授职位）；Humboldt-Universität zu Berlin（柏林洪堡大学）；University of Chicago（芝加哥大学）
Predictive inference for time series: why is split conformal effective despite temporal dependence?
Department of Statistics, University of Chicago（芝加哥大学统计系）；Schools of Industrial and Systems Engineering and Electrical and Computer Engineering, Georgia Tech（佐治亚理工学院工业与系统工程及电气与计算机工程学院）
