Linear thresholding models postulate that the conditional distribution of a response variable in terms of covariates differs on the two sides of a (typically unknown) hyperplane in the covariate space. A key goal in such models is to learn about this separating hyperplane. Exact likelihood or least squares methods to estimate the thresholding parameter involve an indicator function which make them difficult to optimize and are, therefore, often tackled by using a surrogate loss that uses a smooth approximation to the indicator. In this paper, we demonstrate that the resulting estimator is asymptotically normal with a near optimal rate of convergence: $n^{-1}$ up to a log factor, in both classification and regression thresholding models. This is substantially faster than the currently established convergence rates of smoothed estimators for similar models in the statistics and econometrics literatures. We also present a real-data application of our approach to an environmental data set where $CO_2$ emission is explained in terms of a separating hyperplane defined through per-capita GDP and urban agglomeration.
In this paper, we propose an estimator of the generalized maximum mean discrepancy between several distributions, constructed by modifying a naive estimator. Asymptotic normality is obtained for this estimator both under equality of these distributions and under the alternative hypothesis.
We propose an estimator of the kernel-based conditional mean dependence measure obtained from an appropriate modification of a naive estimator based on usual empirical estimators. We then get asymptotic normality of this estimator both under conditional mean independence hypothesis and under the alternative hypothesis. A new test for conditional mean independence of random variables valued into Hilbert spaces is then introduced.
We establish asymptotic normality for estimators of the additive regression components under random censorship. To build our estimators, we couple the marginal integration method (Newey (1994)) with an initial Inverse Probability of Censoring Weighted estimator of the multivariate censored regression function introduced by Carbonez et al. (1995) and Kohler et al. (2002). Asymptotic confidence bands are derived from our result.
In this paper, the maximum spacing method is considered for multivariate observations. Nearest neighbour balls are used as a multidimensional analogue to univariate spacings. A class of information-type measures is used to generalize the concept of maximum spacing estimators. Asymptotic normality of these generalized maximum spacing estimators is proved when the assigned model class is correct, that is the true density is a member of the model class.
We derive asymptotic normality of kernel type deconvolution density estimators. In particular we consider deconvolution problems where the known component of the convolution has a symmetric lambda-stable distribution, 0<lambda<= 2. It turns out that the limit behavior changes if the exponent parameter lambda passes the value one, the case of Cauchy deconvolution.
In the present paper we propose a new estimator of entropy based on smooth estimators of quantile density. The consistency and asymptotic distribution of the proposed estimates are obtained. As a consequence, a new test of normality is proposed. A small power comparison is provided. A simulation study for the comparison, in terms of mean squared error, of all estimators under study is performed.
The main purpose of this paper is to estimate the regression function by using a recursive nonparametric kernel approach. We derive the asymptotic normality for a general class of recursive kernel estimate of the regression function, under strong mixing conditions. Our purpose is to extend the work of Roussas and Tran [17] concerning the Devroye-Wagner estimate.
We derive asymptotic normality of kernel type deconvolution estimators of the density, the distribution function at a fixed point, and of the probability of an interval. We consider the so called super smooth case where the characteristic function of the known distribution decreases exponentially. It turns out that the limit behavior of the pointwise estimators of the density and distribution function is relatively straightforward while the asymptotics of the estimator of the probability of an interval depends in a complicated way on the sequence of bandwidths.
This paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by Good [Biometrica 40 (1953) 237--264]. This new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases.
To enhance the robustness of the classic least sum of squares (LS) of the residuals estimator, Zuo (2022) introduced the least sum of squares of trimmed (LST) residuals estimator. The LST enjoys many desired properties and serves well as a robust alternative to the LS. Its asymptotic properties, including strong and root-n consistency, have been established whereas the asymptotic normality is left unaddressed. This article solves this remained problem.
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
We establish sufficient conditions for the asymptotic normality of kernel density estimators, applied to causal linear random fields. Our conditions on the coefficients of linear random fields are weaker than known results, although our assumption on the bandwidth is not minimal. The proof is based on the $m$-approximation method. As a key step, we prove a central limit theorem for triangular arrays of stationary $m$-dependent random fields with unbounded $m$. We also apply a moment inequality recently established for stationary random fields.
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
The paper concerns the asymptotic distribution of the mixture density estimator, proposed by Oppenheim et al 2006, in the aggregation/disaggregation problem of random parameter AR(1) process. We prove that, under mild conditions on the (semiparametric) form of the mixture density, the estimator is asymptotically normal. The proof is based on the limit theory for the quadratic form in linear random variables developed by Bhansali et al 2007. The moving average representation of the aggregated process is investigated. A small simulation study illustrates the result.
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
Stochastic simulators such as Monte-Carlo estimators are widely used in science and engineering to study physical systems through their probabilistic representation. Global sensitivity analysis aims to identify the input parameters which have the most important impact on the output. A popular tool to perform global sensitivity analysis is the variance-based method which comes from the Hoeffding-Sobol decomposition. Nevertheless, this method requires an important number of simulations and is often unfeasible under reasonable time constraint. Therefore, an approximation of the input/output relat…
We consider linear processes, not necessarily Gaussian, with long, short or negative memory. The memory parameter is estimated semi-parametrically using wavelets from a sample $X_1,...,X_n$ of the process. We treat both the log-regression wavelet estimator and the wavelet Whittle estimator. We show that these estimators are asymptotically normal as the sample size $n\to\infty$ and we obtain an explicit expression for the limit variance. These results are derived from a general result on the asymptotic normality of the empirical scalogram for linear processes, conveniently centered and normaliz…
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
We show that spline and wavelet series regression estimators for weakly dependent regressors attain the optimal uniform (i.e. sup-norm) convergence rate $(n/\log n)^{-p/(2p+d)}$ of Stone (1982), where $d$ is the number of regressors and $p$ is the smoothness of the regression function. The optimal rate is achieved even for heavy-tailed martingale difference errors with finite $(2+(d/p))$th absolute moment for $d/p<2$. We also establish the asymptotic normality of t statistics for possibly nonlinear, irregular functionals of the conditional mean function under weak conditions. The results are p…
In linear models, the generalized least squares (GLS) estimator is applicable when the structure of the error dependence is known. When it is unknown, such structure must be approximated and estimated in a manner that may lead to misspecification. The large-sample analysis of incorrectly-specified GLS (IGLS) estimators requires careful asymptotic manipulations. When performing estimation in the frequency domain, the asymptotic normality of the IGLS estimator, under the so-called Grenander assumptions, has been proved for a broad class of error dependence models. Under the same assumptions, asy…
A multivariable measurement error model $AX \approx B$ is considered. Here $A$ and $B$ are input and output matrices of measurements and $X$ is a rectangular matrix of fixed size to be estimated. The errors in $[A,B]$ are row-wise independent, but within each row the errors may be correlated. Some of the columns are observed without errors and the error covariance matrices may differ from row to row. The total covariance structure of the errors is known up to a scalar factor. The fully weighted total least squares estimator of $X$ is studied. We give conditions for asymptotic normality of the …
The paper discusses the estimation of a continuous density function of the target random field $X_{\bf{i}}$, $\bf{i}\in \mathbb {Z}^N$ which is contaminated by measurement errors. In particular, the observed random field $Y_{\bf{i}}$, $\bf{i}\in \mathbb {Z}^N$ is such that $Y_{\bf{i}}=X_{\bf{i}}+ε_{\bf{i}}$, where the random error $ε_{\bf{i}}$ is from a known distribution and independent of the target random field. Compared to the existing results, the paper is improved in two directions. First, the random vectors in contrast to univariate random variables are investigated. Second, a random fi…
Consider $n$ i.i.d. random elements on $C[0,1]$. We show that, under an appropriate strengthening of the domain of attraction condition, natural estimators of the extreme-value index, which is now a continuous function, and the normalizing functions have a Gaussian process as limiting distribution. A key tool is the weak convergence of a weighted tail empirical process, which makes it possible to obtain the results uniformly on $[0,1]$. Detailed examples are also presented.
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
The problem of estimating the tail index from truncated data is addressed in Chakrabarty and Samorodnitsky (2009). In that paper, a sample based (and hence random) choice of k is suggested, and it is shown that the choice leads to a consistent estimator of the inverse of the tail index. In this paper, the second order behavior of the Hill estimator with that choice of k is studied, under some additional assumptions. In the untruncated situation, it is well known that asymptotic normality of the Hill estimator follows from the assumption of second order regular variation of the underlying distr…
We prove conditional asymptotic normality of a class of quadratic U-statistics that are dominated by their degenerate second order part and have kernels that change with the number of observations. These statistics arise in the construction of estimators in high-dimensional semi- and non-parametric models, and in the construction of nonparametric confidence sets. This is illustrated by estimation of the integral of a square of a density or regression function, and estimation of the mean response with missing data. We show that estimators are asymptotically normal even in the case that the rate…
We prove the asymptotic normality of the kernel density estimator (introduced by Rosenblatt (1956) and Parzen (1962)) in the context of stationary strongly mixing random fields. Our approach is based on the Lindeberg's method rather than on Bernstein's small-block-large-block technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. Our method allows us to consider only minimal conditions on the bandwidth parameter and provides a simple criterion on the (non-uniform) strong mixing coefficients which do not depend on the bandwith.
Statistical analysis of network is an active research area and the literature counts a lot of papers concerned with network models and statistical analysis of networks. However, very few papers deal with missing data in network analysis and we reckon that, in practice, networks are often observed with missing values. In this paper we focus on the Stochastic Block Model with valued edges and consider a MCAR setting by assuming that every dyad (pair of nodes) is sampled identically and independently of the others with probability $ρ> 0$. We prove that maximum likelihood estimators and its variat…
Spectral singularities at non-zero frequencies play an important role in investigating cyclic or seasonal time series. The publication [2] introduced the generalized filtered method-of-moments approach to simultaneously estimate singularity location and long-memory parameters. This paper continues studies of these simultaneous estimators. A wide class of Gegenbauer-type semi-parametric models is considered. Asymptotic normality of several statistics of the cyclic and long-memory parameters is proved. New adjusted estimates are proposed and investigated. The theoretical findings are illustrated…
Let $X_1,...,X_n$ be i.i.d. observations, where $X_i=Y_i+σ_n Z_i$ and the $Y$'s and $Z$'s are independent. Assume that the $Y$'s are unobservable and that they have the density $f$ and also that the $Z$'s have a known density $k.$ Furthermore, let $σ_n$ depend on $n$ and let $σ_n\to 0$ as $n\to\infty.$ We consider the deconvolution problem, i.e. the problem of estimation of the density $f$ based on the sample $X_1,...,X_n.$ A popular estimator of $f$ in this setting is the deconvolution kernel density estimator. We derive its asymptotic normality under two different assumptions on the relation…
It is well known that under general regularity conditions the distribution of the maximum likelihood estimator (MLE) is asymptotically normal. Very recently, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the so-called bounded Wasserstein distance were obtained, where $n$ is the sample size. However, the corresponding bounds on the Kolmogorov distance were only of the order $O(1/n^{1/4})$. In this note, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the Kolmogorov distance ar…
The Latent Block Model (LBM) is a model-based method to cluster simultaneously the $d$ columns and $n$ rows of a data matrix. Parameter estimation in LBM is a difficult and multifaceted problem. Although various estimation strategies have been proposed and are now well understood empirically, theoretical guarantees about their asymptotic behavior is rather sparse and most results are limited to the binary setting. We prove here theoretical guarantees in the valued settings. We show that under some mild conditions on the parameter space, and in an asymptotic regime where $\log(d)/n$ and $\log(n…
This paper develops asymptotic normality results for individual coordinates of robust M-estimators with convex penalty in high-dimensions, where the dimension $p$ is at most of the same order as the sample size $n$, i.e, $p/n\leγ$ for some fixed constant $γ>0$. The asymptotic normality requires a bias correction and holds for most coordinates of the M-estimator for a large class of loss functions including the Huber loss and its smoothed versions regularized with a strongly convex penalty. The asymptotic variance that characterizes the width of the resulting confidence intervals is estimated w…
We investigate the limit behavior of the $L_k$-distance between a decreasing density $f$ and its nonparametric maximum likelihood estimator $\hat{f}_n$ for $k\geq1$. Due to the inconsistency of $\hat{f}_n$ at zero, the case $k=2.5$ turns out to be a kind of transition point. We extend asymptotic normality of the $L_1$-distance to the $L_k$-distance for $1\leq k<2.5$, and obtain the analogous limiting result for a modification of the $L_k$-distance for $k\geq2.5$. Since the $L_1$-distance is the area between $f$ and $\hat{f}_n$, which is also the area between the inverse $g$ of $f$ and the more…
Boltzmann machines (BMs) are a class of binary neural networks for which there have been numerous proposed methods of estimation. Recently, it has been shown that in the fully visible case of the BM, the method of maximum pseudolikelihood estimation (MPLE) results in parameter estimates which are consistent in the probabilistic sense. In this article, we investigate the properties of MPLE for the fully visible BMs further, and prove that MPLE also yields an asymptotically normal parameter estimator. These results can be used to construct confidence intervals and to test statistical hypotheses.…
A key feature of a sequential study is that the actual sample size is a random variable that typically depends on the outcomes collected. While hypothesis testing theory for sequential designs is well established, parameter and precision estimation is less well understood. Even though earlier work has established a number of ad hoc estimators to overcome alleged bias in the ordinary sample average, recent work has shown the sample average to be consistent. Building upon these results, by providing a rate of convergence for the total variation distance, it is established that the asympotic dist…
We consider the estimation of the affine parameter (and power-law exponent) in the preferential attachment model with random initial degrees. We derive the likelihood, and show that the maximum likelihood estimator (MLE) is asymptotically normal and efficient. We also propose a quasi-maximum-likelihood estimator (QMLE) to overcome the MLE's dependence on the history of the initial degrees. To demonstrate the power of our idea, we present numerical simulations.
We prove the local asymptotic mixed normality (LAMN) property for a family of probability measures defined by parametrized diffusion processes with nonsynchronous observations. We assume that observation times of processes are independent of processes and we will study asymptotics when the maximum length of observation intervals goes to zero in probability. We also prove that the quasi-maximum likelihood estimator and the Bayes-type estimator proposed in Ogihara and Yoshida (Stochastic Process. Appl. 124 (2014) 2954-3008) are asymptotically efficient.
We will consider the following stochastic differential equation (SDE): \begin{equation} X_t=X_0+\int_0^tb(X_s,θ_0)ds+σB_t,~~~t\in(0,T], \end{equation} where $\{B_t\}_{t\ge 0}$ is a fractional Brownian motion with Hurst index $H\in(1/2,1)$, $θ_0$ is a parameter that contains a bounded and open convex subset $Θ\subset\mathbb{R}^d$, $\{b(\cdot,θ),θ\inΘ\}$ is a family of drift coefficients with $b(\cdot,θ):\mathbb{R}\rightarrow\mathbb{R}$, and $σ\in\mathbb{R}$ is assumed to be the known diffusion coefficient.
In this paper, we consider an inference problem for the first order autoregressive process with non-zero mean driven by a long memory stationary Gaussian process. Suppose that the covariance function of the noise can be expressed as $|k|^{2H-2}$ times a positive constant when $k$ tends to infinity, and the fractional Gaussian noise and the fractional ARIMA model are special examples that satisfy this assumption. We propose moment estimators and prove the strong consistency, the asymptotic normality and joint asymptotic normality.
Maximum entropy models, motivated by applications in neuron science, are natural generalizations of the $β$-model to weighted graphs. Similar to the $β$-model, each vertex in maximum entropy models is assigned a potential parameter, and the degree sequence is the natural sufficient statistic. Hillar and Wibisono (2013) has proved the consistency of the maximum likelihood estimators. In this paper, we further establish the asymptotic normality for any finite number of the maximum likelihood estimators in the maximum entropy models with three types of edge weights, when the total number of param…
We establish the asymptotic normality of the regression estimator in a fixed-design setting when the errors are given by a field of dependent random variables. The result applies to martingale-difference or strongly mixing random fields. On this basis, a statistical test that can be applied to image analysis is also presented.
We consider the sparse high-dimensional linear regression model $Y=Xb+ε$ where $b$ is a sparse vector. For the Bayesian approach to this problem, many authors have considered the behavior of the posterior distribution when, in truth, $Y=Xβ+ε$ for some given $β$. There have been numerous results about the rate at which the posterior distribution concentrates around $β$, but few results about the shape of that posterior distribution. We propose a prior distribution for $b$ such that the marginal posterior distribution of an individual coordinate $b_i$ is asymptotically normal centered around an …
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the statistical estimation of this index from a finite sample of model outputs: we present two estimators and state a central limit theorem for each. We show that one of these estimators has an optimal asympt…
Let $\mathbf{Y}=\mathbf{X}\boldsΘ\mathbf{Z}'+\bolds{\mathcal {E}}$ be the growth curve model with $\bolds{\mathcal{E}}$ distributed with mean $\mathbf{0}$ and covariance $\mathbf{I}_n\otimes\boldsΣ$, where $\boldsΘ$, $\boldsΣ$ are unknown matrices of parameters and $\mathbf{X}$, $\mathbf{Z}$ are known matrices. For the estimable parametric transformation of the form $\bolds γ=\mathbf{C}\boldsΘ\mathbf{D}'$ with given $\mathbf{C}$ and $\mathbf{D}$, the two-stage generalized least-squares estimator $\hat{\bolds γ}(\mathbf{Y})$ defined in (7) converges in probability to $\boldsγ$ as the sample siz…
We study parametric inference for diffusion processes when observations occur nonsynchronously and are contaminated by market microstructure noise. We construct a quasi-likelihood function and study asymptotic mixed normality of maximum-likelihood- and Bayes-type estimators based on it. We also prove the local asymptotic normality of the model and asymptotic efficiency of our estimator when the diffusion coefficients are constant and noise follows a normal distribution. We conjecture that our estimator is asymptotically efficient even when the latent process is a general diffusion process. An …
We construct a density estimator and an estimator of the distribution function in the uniform deconvolution model. The estimators are based on inversion formulas and kernel estimators of the density of the observations and its derivative. Asymptotic normality and the asymptotic biases are derived.
We introduce a consistent estimator of the extreme value index under random truncation based on a single sample fraction of top observations from truncated and truncation data. We establish the asymptotic normality of the proposed estimator by making use of the weighted tail-copula process framework and we check its finite sample behavior through some simulations. As an application, we provide asymptotic normality results for an estimator of the excess-of-loss reinsurance premium.
