Limit distributions for the greatest convex minorant and its derivative are considered for a general class of stochastic processes including partial sum processes and empirical processes, for independent, weakly dependent and long range dependent data. The results are applied to isotonic regression, isotonic regression after kernel smoothing, estimation of convex regression functions, and estimation of monotone and convex density functions. Various pointwise limit distributions are obtained, and the rate of convergence depends on the self similarity properties and on the rate of convergence of the processes considered.
Wald-type tests are convenient because they allow one to test a wide array of linear and nonlinear restrictions from a single unrestricted estimator; we focus on the problem of implementing Wald-type tests for nonlinear restrictions. We provide examples showing that Wald statistics in non-regular cases can have several asymptotic distributions; the usual critical values based on a chi-square distribution can both lead to under-rejections and over-rejections; indeed, the Wald statistic may diverge under the null hypothesis. We study the asymptotic distribution of Wald-type statistics for the cl…
Shape restrictions have played a central role in economics as both testable implications of theory and sufficient conditions for obtaining informative counterfactual predictions. In this paper we provide a general procedure for inference under shape restrictions in identified and partially identified models defined by conditional moment restrictions. Our test statistics and proposed inference methods are based on the minimum of the generalized method of moments (GMM) objective function with and without shape restrictions. Uniformly valid critical values are obtained through a bootstrap procedu…
The main Theorem of Jain et al.[Jain, K., Singh, S., and Sharma, S. (2011), Re- stricted estimation in multivariate measurement error regression model; JMVA, 102, 2, 264-280] is established in its full generality. Namely, we derive the joint asymp- totic normality of the unrestricted estimator (UE) and the restricted estimators of the matrix of the regression coefficients. The derived result holds under the hypothesized restriction as well as under the sequence of alternative restrictions. In addition, we establish Asymptotic Distributional Risk for the estimators and compare their relative pe…
We present a general methodology for performing statistical inference on the components of a real-valued matrix parameter for which rows and columns are subject to order restrictions. The proposed estimation procedure is based on an iterative algorithm developed by Dykstra and Robertson (1982) for simple order restriction on rows and columns of a matrix. For any order restrictions on rows and columns of a matrix, sufficient conditions are derived for the algorithm to converge in a single application of row and column operations. The new algorithm is applicable to a broad collection of order re…
This paper considers the maximum generalized empirical likelihood (GEL) estimation and inference on parameters identified by high dimensional moment restrictions with weakly dependent data when the dimensions of the moment restrictions and the parameters diverge along with the sample size. The consistency with rates and the asymptotic normality of the GEL estimator are obtained by properly restricting the growth rates of the dimensions of the parameters and the moment restrictions, as well as the degree of data dependence. It is shown that even in the high dimensional time series setting, the …
We consider asymptotically exact inference on the leading canonical correlation directions and strengths between two high dimensional vectors under sparsity restrictions. In this regard, our main contribution is the development of a loss function, based on which, one can operationalize a one-step bias-correction on reasonable initial estimators. Our analytic results in this regard are adaptive over suitable structural restrictions of the high dimensional nuisance parameters, which, in this set-up, correspond to the covariance matrices of the variables of interest. We further supplement the the…
In this paper the asymptotic distribution of estimators is derived in a general regression setting where rank restrictions on a submatrix of the coefficient matrix are imposed and the regressors can include stationary or I(1) processes. Such a setting occurs e.g. in factor models. Rates of convergence are derived and the asymptotic distribution is given for least squares estimators as well as fully-modified estimators. The gains in imposing the rank restrictions are investigated. A number of special cases are discussed including the Johansen results in the case of cointegrated VAR(p) processes…
This paper develops a unified finite-time theory for the ordinary least squares estimation of possibly unstable and even slightly explosive vector autoregressive models under linear restrictions, with the applicable region $ρ(A)\leq 1+c/n$, where $ρ(A)$ is the spectral radius of the transition matrix $A$ in the \VAR(1) representation, $n$ is the time horizon and $c>0$ is a universal constant. The linear restriction framework encompasses various existing models such as banded/network vector autoregressive models. We show that the restrictions reduce the error bounds via not only the reduced dim…
Pattern-mixture models provide a transparent approach for handling missing data, where the full-data distribution is factorized in a way that explicitly shows the parts that can be estimated from observed data alone, and the parts that require identifying restrictions. We introduce a nonparametric estimator of the full-data distribution based on the pattern-mixture model factorization. Our approach uses the empirical observed-data distribution and augments it with a nonparametric estimator of the missing-data distributions under a given identifying restriction. Our results apply to a large cla…
For affine stochastic differential equation with uniformly distributed time delay the local asymptotic properties of the likelihood function are studied. Local asymptotic normality, local asymptotic mixed normality, periodic local asymptotic mixed normality or local asymptotic quadraticity is proved for different values of the parameter. Applications to the asymptotic behaviour of the maximum likelihood estimator of the parameter based on continuous sample are given.
The asymptotic solution to the problem of comparing the means of two heteroscedastic populations, based on two random samples from the populations, hinges on the pivot underpinning the construction of the confidence interval and the test statistic being asymptotically standard Normal, which is known to happen if the two samples are independent and the ratio of the sample sizes converges to a finite positive number. This restriction on the asymptotic behavior of the ratio of the sample sizes carries the risk of rendering the asymptotic justification of the finite sample approximation invalid. I…
In this paper, we seek to establish asymptotic results for selective inference procedures removing the assumption of Gaussianity. The class of selection procedures we consider are determined by affine inequalities, which we refer to as affine selection procedures. Examples of affine selection procedures include post-selection inference along the solution path of the LASSO, as well as post-selection inference after fitting the LASSO at a fixed value of the regularization parameter. We also consider some tests in penalized generalized linear models. Our method of proof adapts a method of Chatter…
In this paper, we develop nonparametric inference on spatial regression models as an extension of Lu and Tj\ostheim(2014), which develops nonparametric inference on density functions of stationary spatial processes under domain expanding and infill (DEI) asymptotics. In particular, we derive multivariate central limit theorems of mean and variance functions of nonparametric spatial regression models. Built upon those results, we propose a method to construct confidence bands for mean and variance functions.
This paper studies an asymptotic framework for conducting inference on parameters of the form $ϕ(θ_0)$, where $ϕ$ is a known directionally differentiable function and $θ_0$ is estimated by $\hat θ_n$. In these settings, the asymptotic distribution of the plug-in estimator $ϕ(\hat θ_n)$ can be readily derived employing existing extensions to the Delta method. We show, however, that the "standard" bootstrap is only consistent under overly stringent conditions -- in particular we establish that differentiability of $ϕ$ is a necessary and sufficient condition for bootstrap consistency whenever the…
We consider the problems of confidence estimation and hypothesis testing on a parameter of signal observed in Gaussian white noise. For these problems we point out lower bounds of asymptotic efficiency in the zone of moderate deviation probabilities. These lower bounds are versions of local asymptotic minimax Hajek-Le Cam lower bound in estimation and the lower bound for Pitman efficiency in hypothesis testing. The lower bounds were obtained for both logarithmic and sharp asymptotic of moderate deviation probabilities.
We describe and examine a test for a general class of shape constraints, such as constraints on the signs of derivatives, U-(S-)shape, symmetry, quasi-convexity, log-convexity, $r$-convexity, among others, in a nonparametric framework using partial sums empirical processes. We show that, after a suitable transformation, its asymptotic distribution is a functional of the standard Brownian motion, so that critical values are available. However, due to the possible poor approximation of the asymptotic critical values to the finite sample ones, we also describe a valid bootstrap algorithm.
The paper presents a systematic theory for asymptotic inference of autocovariances of stationary processes. We consider nonparametric tests for serial correlations based on the maximum (or ${\cal L}^\infty$) and the quadratic (or ${\cal L}^2$) deviations. For these two cases, with proper centering and rescaling, the asymptotic distributions of the deviations are Gumbel and Gaussian, respectively. To establish such an asymptotic theory, as byproducts, we develop a normal comparison principle and propose a sufficient condition for summability of joint cumulants of stationary processes. We adopt …
This work studies the large sample properties of the posterior-based inference in the curved exponential family under increasing dimension. The curved structure arises from the imposition of various restrictions on the model, such as moment restrictions, and plays a fundamental role in econometrics and others branches of data analysis. We establish conditions under which the posterior distribution is approximately normal, which in turn implies various good properties of estimation and inference procedures based on the posterior. In the process we also revisit and improve upon previous results …
Statistical inference based on divergence measures have a long history. Recently, Maji, Ghosh and Basu (2014) have introduced a general family of divergences called the logarithmic super divergence (LSD) family. This family acts as a superfamily for both of the logarithmic power divergence (LPD) family (eg. Renyi, 1961) and the logarithmic density power divergence (LDPD)family introduced by Jones et al. (2001). In this paper we describe the asymptotic properties of the inference procedures resulting from this divergence in discrete models. The properties are well supported by real data example…
This paper shows that the Conditional Quantile Treatment Effect on the Treated can be identified using a combination of (i) a conditional Distributional Difference in Differences assumption and (ii) an assumption on the conditional dependence between the change in untreated potential outcomes and the initial level of untreated potential outcomes for the treated group. The second assumption recovers the unknown dependence from the observed dependence for the untreated group. We also consider estimation and inference in the case where all of the covariates are discrete. We propose a uniform infe…
Many statistical models have likelihoods which are intractable: it is impossible or too expensive to compute the likelihood exactly. In such settings, a common approach is to replace the likelihood with an approximation, and proceed with inference as if the approximate likelihood were the exact likelihood. In this paper, we describe conditions on the approximate likelihood which guarantee that this naive inference with an approximate likelihood has the same first-order asymptotic properties as inference with the exact likelihood. We investigate the implications of these results for inference u…
This paper is concerned with inference in the linear model with dyadic data. Dyadic data is data that is indexed by pairs of "units", for example trade data between pairs of countries. Because of the potential for observations with a unit in common to be correlated, standard inference procedures may not perform as expected. We establish a range of conditions under which a t-statistic with the dyadic-robust variance estimator of Fafchamps and Gubert (2007) is asymptotically normal. Using our theoretical results as a guide, we perform a simulation exercise to study the validity of the normal app…
We derive the precise asymptotic distributional behavior of Gaussian variational approximate estimators of the parameters in a single-predictor Poisson mixed model. These results are the deepest yet obtained concerning the statistical properties of a variational approximation method. Moreover, they give rise to asymptotically valid statistical inference. A simulation study demonstrates that Gaussian variational approximate confidence intervals possess good to excellent coverage properties, and have a similar precision to their exact likelihood counterparts.
For high-dimensional inference problems, statisticians have a number of competing interests. On the one hand, procedures should provide accurate estimation, reliable structure learning, and valid uncertainty quantification. On the other hand, procedures should be computationally efficient and able to scale to very high dimensions. In this note, I show that a very simple data-dependent measure can achieve all of these desirable properties simultaneously, along with some robustness to the error distribution, in sparse sequence models.
Recently, Tibshirani et al. (2016) proposed a method for making inferences about parameters defined by model selection, in a typical regression setting with normally distributed errors. Here, we study the large sample properties of this method, without assuming normality. We prove that the test statistic of Tibshirani et al. (2016) is asymptotically valid, as the number of samples n grows and the dimension d of the regression problem stays fixed. Our asymptotic result holds uniformly over a wide class of nonnormal error distributions. We also propose an efficient bootstrap version of this test…
In this paper we derive some new and practical results on testing and interval estimation problems for the population eigenvalues of a Wishart matrix based on the asymptotic theory for block-wise infinite dispersion of the population eigenvalues. This new type of asymptotic theory has been developed by the present authors in Takemura and Sheena (2005) and Sheena and Takemura (2007a,b) and in these papers it was applied to point estimation problem of population covariance matrix in a decision theoretic framework. In this paper we apply it to some testing and interval estimation problems. We sho…
In this article, we introduce a system of stochastic differential equations (SDEs) consisting of time-dependent covariates and consider both fixed and random effects set-ups. We also allow the functional part associated with the drift function to depend upon unknown parameters. In this general set-up of SDE system we establish consistency and asymptotic normality of the M LE through verification of the regularity conditions required by existing relevant theorems. Besides, we consider the Bayesian approach to learning about the population parameters, and prove consistency and asymptotic normali…
Several interesting models for contingency tables are defined by a system of equality and inequality constraints on a suitable set of marginal log-linear parameters. After reviewing the most common difficulties which are intrinsic to order restricted testing problems, we propose two new families of testing procedures, based on similar attempts appeared in the econometric literature, in order to increase the probability of detecting several relevant violations of the supposed order relations. One set of procedures is based on the decomposition of the log-likelihood ratio when testing the given …
Inspired by sample splitting and the reusable holdout introduced in the field of differential privacy, we consider selective inference with a randomized response. We discuss two major advantages of using a randomized response for model selection. First, the selectively valid tests are more powerful after randomized selection. Second, it allows consistent estimation and weak convergence of selective inference procedures. Under independent sampling, we prove a selective (or privatized) central limit theorem that transfers procedures valid under asymptotic normality without selection to their cor…
We study parametric inference for diffusion processes when observations occur nonsynchronously and are contaminated by market microstructure noise. We construct a quasi-likelihood function and study asymptotic mixed normality of maximum-likelihood- and Bayes-type estimators based on it. We also prove the local asymptotic normality of the model and asymptotic efficiency of our estimator when the diffusion coefficients are constant and noise follows a normal distribution. We conjecture that our estimator is asymptotically efficient even when the latent process is a general diffusion process. An …
Delattre et al. (2013) considered n independent stochastic differential equations (SDEs), where in each case the drift term is associated with a random effect, the distribution of which depends upon unknown parameters. Assuming the independent and identical (iid) situation the authors provide independent proofs of weak consistency and asymptotic normality of the maximum likelihood estimators (MLEs) of the hyper-parameters of their random effects parameters. In this article, as an alternative route to proving consistency and asymptotic normality in the SDE set-up involving random effects, we ve…
We derive asymptotic theory for the plug-in estimate for density level sets under Hausdoff loss. Based on the asymptotic theory, we propose two bootstrap confidence regions for level sets. The confidence regions can be used to perform tests for anomaly detection and clustering. We also introduce a technique to visualize high dimensional density level sets by combining mode clustering and multidimensional scaling.
Ewens-Pitman model has been successfully applied to various fields including Bayesian statistics. There are four important estimators $K_{n},M_{l,n}$,$K_{m}^{(n)},M_{l,m}^{(n)}$. In particular, $M_{1,n}, M_{1,m}^{(n)}$ are related to discovery probability. Their asymptotic behavior, such as large deviation principle, has already been discussed in [4],[1] and [2]. Moderate deviation principle is also discussed in [3] with some speed restriction. In this article, we will apply complex asymptotic analysis to show that this speed restriction is unnecessary.
I propose a new type of confidence interval for correct asymptotic inference after using data to select a model of interest without assuming any model is correctly specified. This hybrid confidence interval is constructed by combining techniques from the selective inference and post-selection inference literatures to yield a short confidence interval across a wide range of data realizations. I show that hybrid confidence intervals have correct asymptotic coverage, uniformly over a large class of probability distributions that do not bound scaled model parameters. I illustrate the use of these …
In this paper, we generalize the property of local asymptotic normality (LAN) to an enlarged neighborhood, under the name of rescaled local asymptotic normality (RLAN). We obtain sufficient conditions for a regular parametric model to satisfy RLAN. We show that RLAN supports the construction of a statistically efficient estimator which maximizes a cubic approximation to the log-likelihood on this enlarged neighborhood. In the context of Monte Carlo inference, we find that this maximum cubic likelihood estimator can maintain its statistical efficiency in the presence of asymptotically increasin…
We discuss estimation problems where a polynomial is observed under Ornstein Uhlenbeck noise over a long time interval. We prove local asymptotic normality (LAN) and specify asymptotically efficient estimators. We apply this to the following problem: feeding noise into the classical (deterministic) Hodgkin Huxley model of neuroscience, we are interested in asymptotically efficient estimation of the parameters of the noise process.
Mendelian randomization (MR) is a method of exploiting genetic variation to unbiasedly estimate a causal effect in presence of unmeasured confounding. MR is being widely used in epidemiology and other related areas of population science. In this paper, we study statistical inference in the increasingly popular two-sample summary-data MR design. We show a linear model for the observed associations approximately holds in a wide variety of settings when all the genetic variants satisfy the exclusion restriction assumption, or in genetic terms, when there is no pleiotropy. In this scenario, we der…
This paper proposes a valid bootstrap-based distributional approximation for M-estimators exhibiting a Chernoff (1964)-type limiting distribution. For estimators of this kind, the standard nonparametric bootstrap is inconsistent. The method proposed herein is based on the nonparametric bootstrap, but restores consistency by altering the shape of the criterion function defining the estimator whose distribution we seek to approximate. This modification leads to a generic and easy-to-implement resampling method for inference that is conceptually distinct from other available distributional approx…
We study convex empirical risk minimization for high-dimensional inference in binary models. Our first result sharply predicts the statistical performance of such estimators in the linear asymptotic regime under isotropic Gaussian features. Importantly, the predictions hold for a wide class of convex loss functions, which we exploit in order to prove a bound on the best achievable performance among them. Notably, we show that the proposed bound is tight for popular binary models (such as Signed, Logistic or Probit), by constructing appropriate loss functions that achieve it. More interestingly…
