We prove the consistency of the Power-Law Fit PLFit method proposed by Clauset et al.(2009) to estimate the power-law exponent in data coming from a distribution function with regularly-varying tail. In the complex systems community, PLFit has emerged as the method of choice to estimate the power-law exponent. Yet, its mathematical properties are still poorly understood.
The difficulty in PLFit is that it is a minimum-distance estimator. It first chooses a threshold that minimizes the Kolmogorov-Smirnov distance between the data points larger than the threshold and the Pareto tail, and then applies the Hill estimator to this restricted data. Since the number of order statistics used is random, the general theory of consistency of power-law exponents from extreme value theory does not apply. Our proof consists in first showing that the Hill estimator is consistent for general intermediate sequences for the number of order statistics used, even when that number is random. Here, we call a sequence intermediate when it grows to infinity, while remaining much smaller than the sample size. The second, and most involved, step is to prove that the optimizer in PLFit is with high probability an intermediate sequence, unless the distribution has a Pareto tail above a certain value. For the latter special case, we give a separate proof.
The aim of this article is to simplify Pfanzagl's proof of consistency for asymptotic maximum likelihood estimators, and to extend it to more general asymptotic M-estimators. The method relies on the existence of a sort of contraction of the parameter space which admits the true parameter as a fixed point. The proofs are short and elementary.
We describe a four-level hierarchy mapping both all discrete estimation problems and all estimators on these problems, such that the hierarchy describes each estimator's consistency guarantees on each problem class. We show that no estimator is consistent for all estimation problems, but that some estimators, such as Maximum A Posteriori, are consistent for the widest possible class of discrete estimation problems. For Maximum Likelihood and Approximate Maximum Likelihood estimators we show that they do not provide consistency on as wide a class, but define a sub-class of problems characterise…
We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon's entropy. As a consequence, an asymptotic 100% confidence interval of entropy is provided.
In finite mixtures of location-scale distributions, if there is no constraint or penalty on the parameters, then the maximum likelihood estimator does not exist because the likelihood is unbounded. To avoid this problem, we consider a penalized likelihood, where the penalty is a function of the minimum of the ratios of the scale parameters and the sample size. It is shown that the penalized maximum likelihood estimator is strongly consistent. We also analyze the consistency of a penalized maximum likelihood estimator where the penalty is imposed on the scale parameters themselves.
In this article, basing on NQD samples, we investigate the fixed design nonparametric regression model, where the errors are pairwise NQD random errors, with fixed design points, and an unknown function. Nonparametric weighted estimator will be introduced and its consistency is studied. As special case, the consistency result for weighted kernel estimators of the model is obtained. This extends the earlier work on independent random and dependent random errors to NQD case.
This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well- specified and misspecified models.
This paper highlights a tension between semiparametric efficiency and bootstrap consistency in the context of a canonical semiparametric estimation problem, namely the problem of estimating the average density. It is shown that although simple plug-in estimators suffer from bias problems preventing them from achieving semiparametric efficiency under minimal smoothness conditions, the nonparametric bootstrap automatically corrects for this bias and that, as a result, these seemingly inferior estimators achieve bootstrap consistency under minimal smoothness conditions. In contrast, several "debi…
In this paper we establish the uniform in bandwidth consistency for the transformation kernel estimator of copulas introduced in [Omelka et al.(2009)]. To this end, we first prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of this estimator from its expectation. We then show that, as n goes to infinity, the bias of the estimator converges to zero uniformly in the bandwidth h, varying over a suitable interval. A practical method of selecting the optimal bandwidth is also presented. Finally, we make conclusive simulation experiments showing the performance of …
Generalized linear mixed models are powerful tools for analyzing clustered data, where the unknown parameters are classically (and most commonly) estimated by the maximum likelihood and restricted maximum likelihood procedures. However, since the likelihood based procedures are known to be highly sensitive to outliers, M-estimators have become popular as a means to obtain robust estimates under possible data contamination. In this paper, we prove that, for sufficiently smooth general loss functions defining the M-estimators in generalized linear mixed models, the tail probability of the deviat…
In 1991 Stute introduced a class of estimators called conditional U-statistics. They can be seen as a generalization of the Nadaraya-Watson estimator, and their strong pointwise consistency to the general regression function has been obtained in the same paper by Stute. Very recently, Gine and Mason introduced the notion of a local U-process, which generalizes that of a local empirical process, and obtained central limit theorems and laws of the iterated logarithm for this class. We apply the methods developed by Einmahl and Mason (2005) and Gine and Mason (2007a,b) to establish uniform in ban…
We consider a one dimensional ballistic random walk evolving in an i.i.d. parametric random environment. We provide a maximum likelihood estimation procedure of the environment parameters based on a single observation of the path till the time it reaches a distant site, and prove that this estimator is consistent as the distant site tends to infinity. We also explore the numerical performances of our estimation procedure.
Consider an $(L,1)$ random walk in an i.i.d. random environment, whose environment involves certain parameter. We get the maximum likelihood estimator(MLE) of the environment parameter which can be written as functionals of a multitype branching process with immigration in a random environment(BPIRE). Because the offspring distributions of the involved multitype BPIRE are of the linear fractional type, the limit invariant distribution of the multitype BPIRE can be computed explicitly. As a result, we get the consistency of the MLE. Our result is a generalization of Comets et al. [Stochastic Pr…
The article introduces spatial long-range dependent models based on the fractional difference operators associated with the Gegenbauer polynomials. The results on consistency and asymptotic normality of a class of minimum contrast estimators of long-range dependence parameters of the models are obtained. A methodology to verify assumptions for consistency and asymptotic normality of minimum contrast estimators is developed. Numerical results are presented to confirm the theoretical findings.
Consider a random vector (X, T), where X is d-dimensional and T is one-dimensional. We suppose that the random variable T is subject to random right censoring and satisfies the $α$-mixing property. The aim of this paper is to study the behavior of the kernel estimator of the relative error regression and to establish its uniform almost sure consistency with rate. Furthermore, we have highlighted the covariance term which measures the dependency. The simulation study shows that the proposed estimator performs well for a finite sample size in different cases.
A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to ze…
The class of observation-driven models (ODMs) includes many models of non-linear time series which, in a fashion similar to, yet different from, hidden Markov models (HMMs), involve hidden variables. Interestingly, in contrast to most HMMs, ODMs enjoy likelihoods that can be computed exactly with computational complexity of the same order as the number of observations, making maximum likelihood estimation the privileged approach for statistical inference for these models. A celebrated example of general order ODMs is the GARCH$(p,q)$ model, for which ergodicity and inference has been studied e…
Estimating the effective dimension reduction (EDR) space, related to the semiparametric regression model introduced by Li \cite{sir}, is based on the estimation of the covariance matrix $Λ$ of the conditional expectation of the vector of predictors given the response. An estimator $\widehatΛ_n$ of $Λ$ based on kernel method was introduced by Zhu and Fang \cite{Asymptotics} who then derived, under some conditions, the asymptotic distribution of $\sqrt{n}\left(\widehatΛ_n-Λ\right)$, as $n\rightarrow +\infty$. In this paper, we obtain, under specified conditions, the almost sure convergence of $\…
We establish asymptotic properties of $M$-estimators, defined in terms of a contrast function and observations from a continuous-time locally stationary process. Using the stationary approximation of the sequence, $θ$-weak dependence, and hereditary properties, we give sufficient conditions on the contrast function that ensure consistency and asymptotic normality of the $M$-estimator. As an example, we obtain consistency and asymptotic normality of a localized least squares estimator for observations from a sequence of time-varying Lévy-driven Ornstein-Uhlenbeck processes. Furthermore, for a s…
In this paper we consider two closely related problems : estimation of eigenvalues and eigenfunctions of the covariance kernel of functional data based on (possibly) irregular measurements, and the problem of estimating the eigenvalues and eigenvectors of the covariance matrix for high-dimensional Gaussian vectors. In Peng and Paul (2007), a restricted maximum likelihood (REML) approach has been developed to deal with the first problem. In this paper, we establish consistency and derive rate of convergence of the REML estimator for the functional data case, under appropriate smoothness conditi…
We first study the drift parameter estimation of the fractional Ornstein-Uhlenbeck process (fOU) with periodic mean for every $\frac{1}{2}<H<1$. More precisely, we extend the consistency proved in \cite{DFW} for $\frac{1}{2}<H<\frac{3}{4}$ to the strong consistency for any $\frac{1}{2}<H<1$ on the one hand, and on the other, we also discuss the asymptotic normality given in \cite{DFW}. In the second main part of the paper, we study the strong consistency and the asymptotic normality of the fOU of the second kind with periodic mean for any $\frac{1}{2}<H<1$.
Efron [J. Roy. Statist. Soc. Ser. B 54 (1992) 83--111] proposed a computationally efficient method, called the jackknife-after-bootstrap, for estimating the variance of a bootstrap estimator for independent data. For dependent data, a version of the jackknife-after-bootstrap method has been recently proposed by Lahiri [Econometric Theory 18 (2002) 79--98]. In this paper it is shown that the jackknife-after-bootstrap estimators of the variance of a bootstrap quantile are consistent for both dependent and independent data. Results from a simulation study are also presented.
We consider population-size-dependent branching processes (PSDBPs) which eventually become extinct with probability one. For these processes, we derive maximum likelihood estimators for the mean number of offspring born to individuals when the current population size is $z\geq 1$. As is standard in branching process theory, an asymptotic analysis of the estimators requires us to condition on non-extinction up to a finite generation $n$ and let $n\to\infty$; however, because the processes become extinct with probability one, we are able to demonstrate that our estimators do not satisfy the clas…
Estimating the innovation probability density is an important issue in any regression analysis. This paper focuses on functional autoregressive models. A residual-based kernel estimator is proposed for the innovation density. Asymptotic properties of this estimator depend on the average prediction error of the functional autoregressive function. Sufficient conditions are studied to provide strong uniform consistency and asymptotic normality of the kernel density estimator.
A change point detection procedure using the method of moment estimators is proposed. The test statistics is based on a suitable $Z$-process. The asymptotic behavior of this process is established under both the null and the alternative hypothesis and the consistency of the test is also proved. An estimator for the change point is proposed and its consistency is derived. Some examples of this method applied to a parametric family of random variables are presented.
Statistical analysis of network is an active research area and the literature counts a lot of papers concerned with network models and statistical analysis of networks. However, very few papers deal with missing data in network analysis and we reckon that, in practice, networks are often observed with missing values. In this paper we focus on the Stochastic Block Model with valued edges and consider a MCAR setting by assuming that every dyad (pair of nodes) is sampled identically and independently of the others with probability $ρ> 0$. We prove that maximum likelihood estimators and its variat…
We introduce and study a local linear nonparametric regression estimator for censorship model. The main goal of this paper is, to establish the uniform almost sure consistency result with rate over a compact set for the new estimate. To support our theoretical result, a simulation study has been done to make comparison with the classical regression estimator.
When analyzing modern machine learning algorithms, we may need to handle kernel density estimation (KDE) with intricate kernels that are not designed by the user and might even be irregular and asymmetric. To handle this emerging challenge, we provide a strong uniform consistency result with the $L^\infty$ convergence rate for KDE on Riemannian manifolds with Riemann integrable kernels (in the ambient Euclidean space). We also provide an $L^1$ consistency result for kernel density estimation on Riemannian manifolds with Lebesgue integrable kernels. The isotropic kernels considered in this pape…
Consider a parametrized family of general hidden Markov models, where both the observed and unobserved components take values in a complete separable metric space. We prove that the maximum likelihood estimator (MLE) of the parameter is strongly consistent under a rather minimal set of assumptions. As special cases of our main result, we obtain consistency in a large class of nonlinear state space models, as well as general results on linear Gaussian state space models and finite state models. A novel aspect of our approach is an information-theoretic technique for proving identifiability, whi…
The preferential attachment (PA) model is a popular way of modeling dynamic social networks, such as collaboration networks. Assuming that the PA function takes a parametric form, we propose and study the maximum likelihood estimator of the parameter. Using a supercritical continuous-time branching process framework, we prove the almost sure consistency and asymptotic normality of this estimator. We also provide an estimator that only depends on the final snapshot of the network and prove its consistency, and its asymptotic normality under general conditions. We compare the performance of the …
Kimura and Yoshida treated a model in which the finite variation part of a two-dimensional semimartingale is expressed by time-integration of latent processes. They proposed a correlation estimator between the latent processes and proved its consistency and asymptotic mixed normality. In this paper, we discuss the confidence interval of the correlation estimator to detect the correlation. %between latent processes. We propose two types of estimators for asymptotic variance of the correlation estimator and prove their consistency in a high frequency setting. Our model includes doubly stochastic…
The Latent Block Model (LBM) is a model-based method to cluster simultaneously the $d$ columns and $n$ rows of a data matrix. Parameter estimation in LBM is a difficult and multifaceted problem. Although various estimation strategies have been proposed and are now well understood empirically, theoretical guarantees about their asymptotic behavior is rather sparse and most results are limited to the binary setting. We prove here theoretical guarantees in the valued settings. We show that under some mild conditions on the parameter space, and in an asymptotic regime where $\log(d)/n$ and $\log(n…
Consistency for constrained maximum likelihood estimation and clustering based on mixtures of elliptically-symmetric distributions under general data generating processes
Active learning seeks to build the best possible model with a budget of labelled data by sequentially selecting the next point to label. However the training set is no longer \textit{iid}, violating the conditions required by existing consistency results. Inspired by the success of Stone's Theorem we aim to regain consistency for weighted averaging estimators under active learning. Based on ideas in \citet{dasgupta2012consistency}, our approach is to enforce a small amount of random sampling by running an augmented version of the underlying active learning algorithm. We generalize Stone's Theo…
In a finite mixture of location-scale distributions maximum likelihood estimator does not exist because of the unboundedness of the likelihood function when the scale parameter of some mixture component approaches zero. In order to study the strong consistency of maximum likelihood estimator, we consider the case that the scale parameters of the component distributions are restricted from below by $c_n$, where $c_n$ is a sequence of positive real numbers which tend to zero as the sample size $n$ increases. We prove that under mild regularity conditions maximum likelihood estimator is strongly …
Penalized estimation principle is fundamental to high-dimensional problems. In the literature, it has been extensively and successfully applied to various models with only structural parameters. As a contrast, in this paper, we apply this penalization principle to a linear regression model with a finite-dimensional vector of structural parameters and a high-dimensional vector of sparse incidental parameters. For the estimators of the structural parameters, we derive their consistency and asymptotic normality, which reveals an oracle property. However, the penalized estimators for the incidenta…
We propose new model selection criteria based on generalized ridge estimators dominating the maximum likelihood estimator under the squared risk and the Kullback-Leibler risk in multivariate linear regression. Our model selection criteria have the following favorite properties: consistency, unbiasedness, uniformly minimum variance. Consistency is proven under an asymptotic structure $\frac{p}{n}\to c$ where $n$ is the sample size and $p$ is the parameter dimension of the response variables. In particular, our proposed class of estimators dominates the maximum likelihood estimator under the squ…
Strong consistency of the quasi-maximum likelihood estimator is given for a general class of multidimensional causal processes based on asyMmetric laplacian innovation.
A continuous-time nonlinear regression model with Lévy-driven linear noise process is considered. Sufficient conditions of consistency and asymptotic normality of the Whittle estimator for the parameter of the noise spectral density are obtained in the paper.
Coordinated Science Laboratory, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校协调科学实验室）；ECE Department, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校电子与计算机工程系）
We consider the problem of identifying parameters of a particular class of Markov chains, called Bernoulli Autoregressive (BAR) processes. The structure of any BAR model is encoded by a directed graph. Incoming edges to a node in the graph indicate that the state of the node at a particular time instant is influenced by the states of the corresponding parental nodes in the previous time instant. The associated edge weights determine the corresponding level of influence from each parental node. In the simplest setup, the Bernoulli parameter of a particular node's state variable is a convex comb…
We point out necessary and sufficient conditions of uniform consistency of nonparametric sets of alternatives for widespread nonparametric tests. Nonparametric sets of alternatives can be defined both in terms of distribution function and in terms of density (or signals in the problem of signal detection in Gaussian white noise). In this part of paper such conditions are provided for $χ^2-$tests with increasing number of cells, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients.
We study theoretical properties of regularized robust M-estimators, applicable when data are drawn from a sparse high-dimensional linear model and contaminated by heavy-tailed distributions and/or outliers in the additive errors and covariates. We first establish a form of local statistical consistency for the penalized regression estimators under fairly mild conditions on the error distribution: When the derivative of the loss function is bounded and satisfies a local restricted curvature condition, all stationary points within a constant radius of the true regression vector converge at the m…
This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…
We study asymptotic properties of maximum likelihood estimators of drift parameters for a jump-type Heston model based on continuous time observations, where the jump process can be any purely non-Gaussian Lévy process of not necessarily bounded variation with a Lévy measure concentrated on $(-1,\infty)$. We prove strong consistency and asymptotic normality for all admissible parameter values except one, where we show only weak consistency and mixed normal (but non-normal) asymptotic behavior. It turns out that the volatility of the price process is a measurable function of the price process. …
Let $\mathbf{Y}=\mathbf{X}\boldsΘ\mathbf{Z}'+\bolds{\mathcal {E}}$ be the growth curve model with $\bolds{\mathcal{E}}$ distributed with mean $\mathbf{0}$ and covariance $\mathbf{I}_n\otimes\boldsΣ$, where $\boldsΘ$, $\boldsΣ$ are unknown matrices of parameters and $\mathbf{X}$, $\mathbf{Z}$ are known matrices. For the estimable parametric transformation of the form $\bolds γ=\mathbf{C}\boldsΘ\mathbf{D}'$ with given $\mathbf{C}$ and $\mathbf{D}$, the two-stage generalized least-squares estimator $\hat{\bolds γ}(\mathbf{Y})$ defined in (7) converges in probability to $\boldsγ$ as the sample siz…
We revisit the classical problem of nonparametric density estimation but impose local differential privacy constraints. Under such constraints, the original multivariate data $X_1,\ldots,X_n \in \mathbb{R}^d$ cannot be directly observed, and all estimators are functions of the randomised output of a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and in this work we propose to add Laplace distributed noise to a discretisation of the location of a vector $X_i$. Based on these randomised data, we design a novel estimator of the density function, …
This paper considers extensions of minimum-disparity estimators to the problem of estimating parameters in a regression model that is conditionally specified; that is where a parametric model describes the distribution of a response $y$ conditional on covariates $x$ but does not specify the distribution of $x$. We define these estimators by estimating a non-parametric conditional density estimates and minimizing a disparity between this estimate and the parametric model averaged over values of $x$. The consistency and asymptotic normality of such estimators is demonstrated for a broad class of…
