The present manuscript studies signal detection by likelihood ratio tests in a number of spiked random matrix models, including but not limited to Gaussian mixtures and spiked Wishart covariance matrices. We work directly with multi-spiked cases in these models and with flexible priors on the signal component that allow dependence across spikes. We derive asymptotic normality for the log-likelihood ratios when the signal-to- noise ratios are below certain thresholds. In addition, we show that the variances of the log-likelihood ratios can be asymptotically decomposed as the sums of those of a collection of statistics which we call bipartite signed cycles.
We consider two types of spiked multivariate F distributions: a scaled distribution with the scale matrix equal to a rank-one perturbation of the identity, and a distribution with trivial scale, but rank-one non-centrality. The norm of the rank-one matrix (spike) parameterizes the joint distribution of the eigenvalues of the corresponding F matrix. We show that, for a spike located above a phase transition threshold, the asymptotic behavior of the log ratio of the joint density of the eigenvalues of the F matrix to their joint density under a local deviation from this value depends only on the…
These expository notes serve as a reference for an accompanying post Morales-Jimenez et al. [2018]. In the spiked covariance model, we develop results on asymptotic normality of sample leading eigenvalues and certain projections of the corresponding sample eigenvectors. The results parallel those of Paul [2007], but are given using the non-Gaussian model of Bai and Yao [2008]. The results are not new, and citations are given, but proofs are collected and organized as a point of departure for Morales-Jimenez et al. [2018].
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio of the spiked model against an "all noise" null model of reference, and show it has asymptotically Gaussian fluctuations in a region below---but in general not up to---the so-called BBP threshold from random matrix theory. Our result parallels earlier findings of Onatski et al.\ …
This paper aims to test the number of spikes in a generalized spiked covariance matrix, the spiked eigenvalues of which may be extremely larger or smaller than the non-spiked ones. For a high-dimensional problem, we first propose a general test statistic and derive its central limit theorem by random matrix theory without a Gaussian population constraint. We then apply the result to estimate the noise variance and test the equality of the smallest roots in generalized spiked models. Simulation studies showed that the proposed test method was correctly sized, and the power outcomes showed the r…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
Using the Coulomb Fluid method, this paper derives central limit theorems (CLTs) for linear spectral statistics of three "spiked" Hermitian random matrix ensembles. These include Johnstone's spiked model (i.e., central Wishart with spiked correlation), non-central Wishart with rank-one non-centrality, and a related class of non-central $F$ matrices. For a generic linear statistic, we derive simple and explicit CLT expressions as the matrix dimensions grow large. For all three ensembles under consideration, we find that the primary effect of the spike is to introduce an $O(1)$ correction term t…
In this paper, we generalize the property of local asymptotic normality (LAN) to an enlarged neighborhood, under the name of rescaled local asymptotic normality (RLAN). We obtain sufficient conditions for a regular parametric model to satisfy RLAN. We show that RLAN supports the construction of a statistically efficient estimator which maximizes a cubic approximation to the log-likelihood on this enlarged neighborhood. In the context of Monte Carlo inference, we find that this maximum cubic likelihood estimator can maintain its statistical efficiency in the presence of asymptotically increasin…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does not require prior knowledge about the distribution of the signal or the noise. When the noise is Gaussian, the proposed test is optimal in the sense that its error matches that of the likelihood ratio test, which minimizes the sum of the Type-I and Type-II errors. If the density of…
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
We develop a theory of local asymptotic normality in the quantum domain based on a novel quantum analogue of the log-likelihood ratio. This formulation is applicable to any quantum statistical model satisfying a mild smoothness condition. As an application, we prove the asymptotic achievability of the Holevo bound for the local shift parameter.
Variational methods for parameter estimation are an active research area, potentially offering computationally tractable heuristics with theoretical performance bounds. We build on recent work that applies such methods to network data, and establish asymptotic normality rates for parameter estimates of stochastic blockmodel data, by either maximum likelihood or variational estimation. The result also applies to various sub-models of the stochastic blockmodel found in the literature.
Distributed Online Estimation of Spiked Eigenvalues with Adaptive Weighting under Persistent Aspect Ratio Heterogeneity
School of Mathematical Sciences, Tiangong University（天津工业大学数学科学学院）；KLASMOE and School of Mathematics & Statistics, Northeast Normal University（东北师范大学数学与统计学院）
Generalized likelihoods are commonly used to obtain consistent estimators with attractive computational and robustness properties. Formally, any generalized likelihood can be used to define a generalized posterior distribution, but an arbitrarily defined "posterior" cannot be expected to appropriately quantify uncertainty in any meaningful sense. In this article, we provide sufficient conditions under which generalized posteriors exhibit concentration, asymptotic normality (Bernstein-von Mises), an asymptotically correct Laplace approximation, and asymptotically correct frequentist coverage. W…
We consider the estimation of the affine parameter (and power-law exponent) in the preferential attachment model with random initial degrees. We derive the likelihood, and show that the maximum likelihood estimator (MLE) is asymptotically normal and efficient. We also propose a quasi-maximum-likelihood estimator (QMLE) to overcome the MLE's dependence on the history of the initial degrees. To demonstrate the power of our idea, we present numerical simulations.
It is well known that under general regularity conditions the distribution of the maximum likelihood estimator (MLE) is asymptotically normal. Very recently, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the so-called bounded Wasserstein distance were obtained, where $n$ is the sample size. However, the corresponding bounds on the Kolmogorov distance were only of the order $O(1/n^{1/4})$. In this note, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the Kolmogorov distance ar…
Approximations to the modified signed likelihood ratio statistic are asymptotically standard normal with error of order $n^{-1}$, where $n$ is the sample size. Proofs of this fact generally require that the sufficient statistic of the model be written as $(\hatθ,a)$, where $\hatθ$ is the maximum likelihood estimator of the parameter $θ$ of the model and $a$ is an ancillary statistic. This condition is very difficult or impossible to verify for many models. However, calculation of the statistics themselves does not require this condition. The goal of this paper is to provide conditions under wh…
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how asymptotic normality does--or does not--hold for common severity distributions in operational risk models. We then apply these results to evaluate errors caused by failure of asymptotic normality in constructing confidence intervals around the MLE fitted parameters.
This paper establishes the strong consistency and asymptotic normality of the quasi-maximum likelihood estimator (QMLE) for a GARCH process with periodically time-varying parameters. We first give a necessary and sufficient condition for the existence of a strictly periodically stationary solution for the periodic GARCH (P-GARCH) equation. As a result, it is shown that the moment of some positive order of the P-GARCH solution is finite, under which we prove the strong consistency and asymptotic normality (CAN) of the QMLE without any condition on the moments of the underlying process.
For affine stochastic differential equation with uniformly distributed time delay the local asymptotic properties of the likelihood function are studied. Local asymptotic normality, local asymptotic mixed normality, periodic local asymptotic mixed normality or local asymptotic quadraticity is proved for different values of the parameter. Applications to the asymptotic behaviour of the maximum likelihood estimator of the parameter based on continuous sample are given.
We prove the strong consistency and the asymptotic normality of the maximum likelihood estimator of the parameters of a general conditionally heteroscedastic model with $α$-stable innovations. Then, we relax the assumptions and only suppose that the innovation process converges in distribution toward a stable process. Using a pseudo maximum likelihood estimator with a stable density, we also obtain the strong consistency and the asymptotic normality of the estimator. This framework seems relevant for financial data exhibiting heavy tails. We apply this method to several financial index and com…
Maximum entropy models, motivated by applications in neuron science, are natural generalizations of the $β$-model to weighted graphs. Similar to the $β$-model, each vertex in maximum entropy models is assigned a potential parameter, and the degree sequence is the natural sufficient statistic. Hillar and Wibisono (2013) has proved the consistency of the maximum likelihood estimators. In this paper, we further establish the asymptotic normality for any finite number of the maximum likelihood estimators in the maximum entropy models with three types of edge weights, when the total number of param…
The Latent Block Model (LBM) is a model-based method to cluster simultaneously the $d$ columns and $n$ rows of a data matrix. Parameter estimation in LBM is a difficult and multifaceted problem. Although various estimation strategies have been proposed and are now well understood empirically, theoretical guarantees about their asymptotic behavior is rather sparse and most results are limited to the binary setting. We prove here theoretical guarantees in the valued settings. We show that under some mild conditions on the parameter space, and in an asymptotic regime where $\log(d)/n$ and $\log(n…
We study local asymptotic properties of likelihood ratios of certain Heston models. We distinguish three cases: subcritical, critical and supercritical models. For the drift parameters, local asymptotic normality is proved in the subcritical case, only local asymptotic quadraticity is shown in the critical case, while in the supercritical case not even local asymptotic quadraticity holds. For certain submodels, local asymptotic normality is proved in the critical case, and local asymptotic mixed normality is shown in the supercritical case. As a consequence, asymptotically optimal (randomized)…
The problem of characterizing a multivariate distribution of a random vector using examination of univariate combinations of vector components is an essential issue of multivariate analysis. The likelihood principle plays a prominent role in developing powerful statistical inference tools. In this context, we raise the question: can the univariate likelihood function based on a random vector be used to provide the uniqueness in reconstructing the vector distribution? In multivariate normal (MN) frameworks, this question links to a reverse of Cochran's theorem that concerns the distribution of …
We consider the problem of estimating a large rank-one tensor ${\boldsymbol u}^{\otimes k}\in({\mathbb R}^{n})^{\otimes k}$, $k\ge 3$ in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio $λ_{Bayes}= O(1)$ above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably no polynomial-time algorithm is known that achieved this goal unless $λ\ge C n^{(k-2)/4}$ and even powerful semidefinite programming relaxations appear to fail for $1\ll λ\ll n^{(k-2)/4}$. In order to elucidate this behavior, we consider the maximum l…
We consider a stable Cox--Ingersoll--Ross process driven by a standard Wiener process and a spectrally positive strictly stable Lévy process, and we study asymptotic properties of the maximum likelihood estimator (MLE) for its growth rate based on continuous time observations. We distinguish three cases: subcritical, critical and supercritical. In all cases we prove strong consistency of the MLE in question, in the subcritical case asymptotic normality, and in the supercritical case asymptotic mixed normality are shown as well. In the critical case the description of the asymptotic behavior of…
If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically distributed data. We do not need the law of large numbers (LLN) or the central limit theorem (CLT). We do not need sample size going to infinity or anything going to infinity. Presented here is a combination of Le Cam style theory involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style theory involving derivatives and…
We consider the estimation of parametric fractional time series models in which not only is the memory parameter unknown, but one may not know whether it lies in the stationary/invertible region or the nonstationary or noninvertible regions. In these circumstances, a proof of consistency (which is a prerequisite for proving asymptotic normality) can be difficult owing to nonuniform convergence of the objective function over a large admissible parameter space. In particular, this is the case for the conditional sum of squares estimate, which can be expected to be asymptotically efficient under …
This article provides an introduction to the asymptotic analysis of covariance parameter estimation for Gaussian processes. Maximum likelihood estimation is considered. The aim of this introduction is to be accessible to a wide audience and to present some existing results and proof techniques from the literature. The increasing-domain and fixed-domain asymptotic settings are considered. Under increasing-domain asymptotics, it is shown that in general all the components of the covariance parameter can be estimated consistently by maximum likelihood and that asymptotic normality holds. In contr…
This paper presents asymptotic properties of the maximum pseudo-likelihood estimator of a vector $\Vectθ$ parameterizing a stationary Gibbs point process. Sufficient conditions, expressed in terms of the local energy function defining a Gibbs point process, to establish strong consistency and asymptotic normality results of this estimator depending on a single realization, are presented.These results are general enough to no longer require the local stability and the linearity in terms of the parameters of the local energy function. We consider characteristic examples of such models, the Lenna…
We study the problem of parameter estimation for stochastic differential equations with small noise and fast oscillating parameters. Depending on how fast the intensity of the noise goes to zero relative to the homogenization parameter, we consider three different regimes. For each regime, we construct the maximum likelihood estimator and we study its consistency and asymptotic normality properties. A simulation study for the first order Langevin equation with a two scale potential is also provided.
In this note, we propose a new approach for the proof of the consistency and normality of the maximum likelihood estimator for nonlinear AR processes with markov-switching under the assumptions of uniform exponential forgetting of the prediction filter and $α$-mixing property. We show that in the linear and Gaussian case our assumptions are fully satisfied.
The paper studies large sample asymptotic properties of the Maximum Likelihood Estimator (MLE) for the parameter of a continuous time Markov chain, observed in white noise. Using the method of weak convergence of likelihoods due to I.Ibragimov and R.Khasminskii, consistency, asymptotic normality and convergence of moments are established for MLE under certain strong ergodicity conditions of the chain.
A trigonometrically approximated maximum likelihood estimation for $α$-stable laws is proposed. The estimator solves the approximated likelihood equation, which is obtained by projecting a true score function on the space spanned by trigonometric functions. The projected score is expressed only by real and imaginary parts of the characteristic function and their derivatives, so that we can explicitly construct the targeting estimating equation. We study the asymptotic properties of the proposed estimator and show consistency and asymptotic normality. Furthermore, as the number of trigonometric…
We study local asymptotic normality of M-estimates of convex minimization in an infinite dimensional parameter space. The objective function of M-estimates is not necessary differentiable and is possibly subject to convex constraints. In the above circumstance, narrow convergence with respect to uniform convergence fails to hold, because of the strength of it's topology. A new approach we propose to the lack-of-uniform-convergence is based on Mosco-convergence that is weaker topology than uniform convergence. By applying narrow convergence with respect to Mosco topology, we develop an infinite…
There is a difficulty in finding an estimate of variance of the profile likelihood estimator in the joint model of longitudinal and survival data. We solve the difficulty by introducing the ``statistical generalized derivative''. The derivative is used to show the asymptotic normality of the estimator without assuming the second derivative of the density function in the model exists.
The goal of this paper is to describe the application of quasi-likelihood estimating equations for spatially correlated binary data. In this paper, a logistic function is used to model the marginal probability of binary responses in terms of parameters of interest. With mild assumptions on the correlations, the Leonov-Shiryaev formula combined with a comparison of characteristic functions can be used to establish asymptotic normality for linear combinations of the binary responses. The consistency and asymptotic normality for quasi-likelihood estimates can then be derived. By modeling spatial …
We prove the local asymptotic mixed normality (LAMN) property for a family of probability measures defined by parametrized diffusion processes with nonsynchronous observations. We assume that observation times of processes are independent of processes and we will study asymptotics when the maximum length of observation intervals goes to zero in probability. We also prove that the quasi-maximum likelihood estimator and the Bayes-type estimator proposed in Ogihara and Yoshida (Stochastic Process. Appl. 124 (2014) 2954-3008) are asymptotically efficient.
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
We investigate a semiparametric regression model where one gets noisy non linear non invertible functions of the observations. We focus on the application to bearings-only tracking. We first investigate the least squares estimator and prove its consistency and asymptotic normality under mild assumptions. We study the semiparametric likelihood process and prove local asymptotic normality of the model. This allows to define the efficient Fisher information as a lower bound for the asymptotic variance of regular estimators, and to prove that the parametric likelihood estimator is regular and asym…
A parameter estimation problem is considered for a diagonaliazable stochastic evolution equation using a finite number of the Fourier coefficients of the solution. The equation is driven by additive noise that is white in space and fractional in time with the Hurst parameter $H\geq 1/2$. The objective is to study asymptotic properties of the maximum likelihood estimator as the number of the Fourier coefficients increases. A necessary and sufficient condition for consistency and asymptotic normality is presented in terms of the eigenvalues of the operators in the equation.
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a long established result. Explicit bounds for the distributional distance between the distribution of the MLE and the normal distribution have recently been obtained for the case of independent random variables. In this paper, a local dependence structure is introduced between the random variables and we give upper bounds which are specified for the Wasserstein metric.
