Convergence of Estimators in LLS Analysis

Convergence Rates for a Class of Estimators Based on Stein's Method

Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a test function along the sample path of a Markov chain, where gradient information enables convergence rate improvement at the cost of a linear system which must be solved. The contribution of this paper is to establish theoretical bounds on convergence rates for a class of estimators based on Stein's method. Our analysis accounts for (i) the degree of smoothness of the sampling distribution and test…

Weak convergence of a pseudo maximum likelihood estimator for the extremal index

The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…

Convergence of Chao Unseen Species Estimator

Support size estimation and the related problem of unseen species estimation have wide applications in ecology and database analysis. Perhaps the most used support size estimator is the Chao estimator. Despite its wide spread use, little is known about its theoretical properties. We analyze the Chao estimator and show that its worst case mean squared error (MSE) is smaller than the MSE of the plug-in estimator by a factor of $\mathcal{O} ((k/n)^4)$, where $k$ is the maximum support size and $n$ is the number of samples. Our main technical contribution is a new method to analyze rational estima…

Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators

We derive the $l_{\infty}$ convergence rate simultaneously for Lasso and Dantzig estimators in a high-dimensional linear regression model under a mutual coherence assumption on the Gram matrix of the design and two different assumptions on the noise: Gaussian noise and general noise with finite variance. Then we prove that simultaneously the thresholded Lasso and Dantzig estimators with a proper choice of the threshold enjoy a sign concentration property provided that the non-zero components of the target vector are not too small.

非参数最小二乘估计的局部收敛速率及其在迁移学习中的应用

Local convergence rates of the nonparametric least squares estimator with applications to transfer learning

本文研究了非参数最小二乘估计在加权一致范数下的局部收敛速率，证明其极小极大最优性，并将所得速率应用于协变量偏移下的迁移学习分析。

Naive Penalized Spline Estimators of Derivatives Achieve Optimal Rates of Convergence

This paper studies the asymptotic behavior of penalized spline estimates of derivatives. In particular, we show that simply differentiating the penalized spline estimator of the mean regression function itself to estimate the corresponding derivative achieves the optimal L2 rate of convergence.

带移民的超临界连续状态和连续时间分支过程条件最小二乘估计量的稳定收敛

Stable convergence of conditional least squares estimators for supercritical continuous state and continuous time branching processes with immigration

本文证明了基于离散时间观测的带移民超临界连续状态与连续时间分支过程漂移参数的条件最小二乘估计量具有稳定收敛性。

A Short Review of Ergodicity and Convergence of Markov chain Monte Carlo Estimators

This short note reviews the basic theory for quantifying both the asymptotic and preasymptotic convergence of Markov chain Monte Carlo estimators.

Moment convergence of the generalized maximum composite likelihood estimators for determinantal point processes

The maximum composite likelihood estimator for parametric models of determinantal point processes (DPPs) is discussed. Since the joint intensities of these point processes are given by determinant of positive definite kernels, we have the explicit form of the joint intensities for every order. This fact enables us to consider the generalized maximum composite likelihood estimator for any order. This paper introduces the two step generalized composite likelihood estimator and shows the moment convergence of the estimator under a stationarity. Moreover, our results can yield information criteria…

最近邻估计量收敛速率研究：在平滑回归函数情况下的表现

Rates of convergence for nearest neighbor estimators with the smoother regression function

本文研究了在(p,C)-光滑回归函数情况下，最近邻估计量的收敛速率，证明当p在(1,1.5]范围内时，最优收敛速率可被最近邻估计量实现。

Handy sufficient conditions for the convergence of the maximum likelihood estimator in observation-driven models

This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…

Uniform convergence of convolution estimators for the response density in nonparametric regression

We consider a nonparametric regression model $Y=r(X)+\varepsilon$ with a random covariate $X$ that is independent of the error $\varepsilon$. Then the density of the response $Y$ is a convolution of the densities of $\varepsilon$ and $r(X)$. It can therefore be estimated by a convolution of kernel estimators for these two densities, or more generally by a local von Mises statistic. If the regression function has a nowhere vanishing derivative, then the convolution estimator converges at a parametric rate. We show that the convergence holds uniformly, and that the corresponding process obeys a …

Optimal Uniform Convergence Rates and Asymptotic Normality for Series Estimators Under Weak Dependence and Weak Conditions

We show that spline and wavelet series regression estimators for weakly dependent regressors attain the optimal uniform (i.e. sup-norm) convergence rate $(n/\log n)^{-p/(2p+d)}$ of Stone (1982), where $d$ is the number of regressors and $p$ is the smoothness of the regression function. The optimal rate is achieved even for heavy-tailed martingale difference errors with finite $(2+(d/p))$th absolute moment for $d/p<2$. We also establish the asymptotic normality of t statistics for possibly nonlinear, irregular functionals of the conditional mean function under weak conditions. The results are p…

$L_p$ and almost sure convergence of estimation on heavy tail index under random censoring

In this paper, we prove $L_p,\ p\geq 2$ and almost sure convergence of tail index estimator mentioned in \cite{grama2008} under random censoring and several assumptions. $p$th moment of the error of the estimator is proved to be of order $O\left(\frac{1}{\log^{mκ/2}n}\right)$ with given assumptions. We also perform several finite sample simulations to quantify performance of this estimator. Finite sample results show that the proposed estimator is effective in finding underlying tail index even when censor rate is high.

贝叶斯估计在一般损失函数下的弱收敛性

Weak convergence of Bayes estimators under general loss functions

本文研究了在一般损失函数下贝叶斯估计的弱收敛性，建立了统一理论框架，并推导了后验渐近正态性的条件，同时提供了新的一致性结果和实验验证。

On adaptivity of wavelet thresholding estimators with negatively super-additive dependent noise

This paper considers the nonparametric regression model with negatively super-additive dependent (NSD) noise and investigates the convergence rates of thresholding estimators. It is shown that the term-by-term thresholding estimator achieves nearly optimal and the block thresholding estimator attains optimal (or nearly optimal) convergence rates over Besov spaces. Additionally, some numerical simulations are implemented to substantiate the validity and adaptivity of the thresholding estimators with the presence of NSD noise.

半参数多元混合模型中最大平滑对数似然估计量的收敛速率

Rates of Convergence of Maximum Smoothed Log-Likelihood Estimators for Semi-Parametric Multivariate Mixtures

本文首次为半参数有限混合模型中基于平滑对数似然的最大估计量建立了严格理论保证，证明其一致性并推导有限维与无限维参数的收敛速率，弥合了实用算法与统计理论之间的差距。

Asymptotic Properties of Penalized Spline Estimators in Concave Extended Linear Models: Rates of Convergence

This paper develops a general theory on rates of convergence of penalized spline estimators for function estimation when the likelihood functional is concave in candidate functions, where the likelihood is interpreted in a broad sense that includes conditional likelihood, quasi-likelihood, and pseudo-likelihood. The theory allows all feasible combinations of the spline degree, the penalty order, and the smoothness of the unknown functions. According to this theory, the asymptotic behaviors of the penalized spline estimators depends on interplay between the spline knot number and the penalty pa…

On Integrated $L^{1}$ Convergence Rate of an Isotonic Regression Estimator for Multivariate Observations

We consider a general monotone regression estimation where we allow for independent and dependent regressors. We propose a modification of the classical isotonic least squares estimator and establish its rate of convergence for the integrated $L_1$-loss function. The methodology captures the shape of the data without assuming additivity or a parametric form for the regression function. Furthermore, the degree of smoothing is chosen automatically and no auxiliary tuning is required for the theoretical analysis. Some simulations and two real data illustrations complement the study of the propose…

On convex regression estimators

A new nonparametric estimator of a convex regression function in any dimension is proposed and its convergence properties are studied. We start by using any estimator of the regression function and we \emph{convexify} it by taking the convex envelope of a sample of the approximation obtained. We prove that the uniform rate of convergence of the estimator is maintained after the convexification is applied. The finite sample properties of the new estimator are investigated by means of a simulation study and the application of the new method is demonstrated in examples.

基于稳定化技术的Tsallis熵估计量的渐近正态性与收敛速率

Asymptotic Normality and Convergence Rates for Tsallis Entropy Estimators via Stabilization Techniques

该研究针对一般度量测度空间上的Tsallis熵估计量，采用稳定化技术建立其渐近正态性与收敛速率，扩展了Shannon和Rényi熵的相关结果，为复杂空间和高维场景的非参数统计推断提供了有效框架。

Cube root weak convergence of empirical estimators of a density level set

Given $n$ independent random vectors with common density $f$ on $\mathbb{R}^d$, we study the weak convergence of three empirical-measure based estimators of the convex $λ$-level set $L_λ$ of $f$, namely the excess mass set, the minimum volume set and the maximum probability set, all selected from a class of convex sets $\mathcal{A}$ that contains $L_λ$. Since these set-valued estimators approach $L_λ$, even the formulation of their weak convergence is non-standard. We identify the joint limiting distribution of the symmetric difference of $L_λ$ and each of the three estimators, at rate $n^{-1/…

Uniform convergence and asymptotic confidence bands for model-assisted estimators of the mean of sampled functional data

When the study variable is functional and storage capacities are limited or transmission costs are high, selecting with survey sampling techniques a small fraction of the observations is an interesting alternative to signal compression techniques, particularly when the goal is the estimation of simple quantities such as means or totals. We extend, in this functional framework, model-assisted estimators with linear regression models that can take account of auxiliary variables whose totals over the population are known. We first show, under weak hypotheses on the sampling design and the regular…

二元Archimax Copulas的可识别性、收敛性和非参数估计

Identifiability, Convergence and Nonparametric Estimation of Bivariate Archimax Copulas

研究二元Archimax Copulas的可识别性等问题，通过变换生成元和相依函数解决可识别性，利用收敛等价关系构造非参数估计量，证明其强一致性，模拟研究表明CFG型估计量性能优，还给出实际数据应用。

Estimating the geometric median in Hilbert spaces with stochastic gradient algorithms: $L^{p}$ and almost sure rates of convergence

The geometric median, also called $L^{1}$-median, is often used in robust statistics. Moreover, it is more and more usual to deal with large samples taking values in high dimensional spaces. In this context, a fast recursive estimator has been introduced by Cardot, Cenac and Zitt. This work aims at studying more precisely the asymptotic behavior of the estimators of the geometric median based on such non linear stochastic gradient algorithms. The $L^{p}$ rates of convergence as well as almost sure rates of convergence of these estimators are derived in general separable Hilbert spaces. Moreove…

积分扩散过程非参数估计量的一致收敛速度

Uniform Convergence Rate of the Nonparametric Estimator for Integrated Diffusion Processes

本文研究积分扩散过程系数的非参数估计，首次在观测区间趋于消失、长时间跨度及无界支撑条件下，为 Nadaraya-Watson 估计量建立一致收敛速度。

Optimal rate of convergence for nonparametric change-point estimators for nonstationary sequences

Let $(X_i)_{i=1,...,n}$ be a possibly nonstationary sequence such that $\mathscr{L}(X_i)=P_n$ if $i\leq nθ$ and $\mathscr{L}(X_i)=Q_n$ if $i>nθ$, where $0<θ<1$ is the location of the change-point to be estimated. We construct a class of estimators based on the empirical measures and a seminorm on the space of measures defined through a family of functions $\mathcal{F}$. We prove the consistency of the estimator and give rates of convergence under very general conditions. In particular, the $1/n$ rate is achieved for a wide class of processes including long-range dependent sequences and even no…

Uniformly root-$N$ consistent density estimators for weakly dependent invertible linear processes

Convergence rates of kernel density estimators for stationary time series are well studied. For invertible linear processes, we construct a new density estimator that converges, in the supremum norm, at the better, parametric, rate $n^{-1/2}$. Our estimator is a convolution of two different residual-based kernel estimators. We obtain in particular convergence rates for such residual-based kernel estimators; these results are of independent interest.

On minimax nonparametric estimation of signal in Gaussian noise

For the problem of nonparametric estimation of signal in Gaussian noise we point out the strong asymptotically minimax estimators on maxisets for linear estimators (see \cite{ker93,rio}). It turns out that the order of rates of convergence of Pinsker estimator on this maxisets is worse than the order of rates of convergence for the class of linear estimators considered on this maxisets. We show that balls in Sobolev spaces are maxisets for Pinsker estimators.

Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap

Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle l…

Convergence of least squares estimators in the adaptive Wynn algorithm for a class of nonlinear regression models

The paper continues the authors' work on the adaptive Wynn algorithm in a nonlinear regression model. In the present paper it is shown that if the mean response function satisfies a condition of `saturated identifiability', which was introduced by Pronzato \cite{Pronzato}, then the adaptive least squares estimators are strongly consistent. The condition states that the regression parameter is identifiable under any saturated design, i.e., the values of the mean response function at any $p$ distinct design points determine the parameter point uniquely where, typically, $p$ is the dimension of t…

Maximum likelihood estimator for skew Brownian motion: the convergence rate

We give a thorough description of the asymptotic property of the maximum likelihood estimator (MLE) of the skewness parameter of a Skew Brownian Motion (SBM). Thanks to recent results on the Central Limit Theorem of the rate of convergence of estimators for the SBM, we prove a conjecture left open that the MLE has asymptotically a mixed normal distribution involving the local time with a rate of convergence of order $1/4$. We also give a series expansion of the MLE and study the asymptotic behavior of the score and its derivatives, as well as their variation with the skewness parameter. In par…

Bounds in $L^1$ Wasserstein distance on the normal approximation of general M-estimators

We derive quantitative bounds on the rate of convergence in $L^1$ Wasserstein distance of general M-estimators, with an almost sharp (up to a logarithmic term) behavior in the number of observations. We focus on situations where the estimator does not have an explicit expression as a function of the data. The general method may be applied even in situations where the observations are not independent. Our main application is a rate of convergence for cross validation estimation of covariance parameters of Gaussian processes.

Revisiting Révész's stochastic approximation method for the estimation of a regression function

In a pioneer work, Révész (1973) introduces the stochastic approximation method to build up a recursive kernel estimator of the regression function $x\mapsto E(Y|X=x)$. However, according to Révész (1977), his estimator has two main drawbacks: on the one hand, its convergence rate is smaller than that of the nonrecursive Nadaraya-Watson's kernel regression estimator, and, on the other hand, the required assumptions on the density of the random variable $X$ are stronger than those usually needed in the framework of regression estimation. We first come back on the study of the convergence rate o…

通过适应最优输运的依赖性度量：稳定性与收敛速率

Dependence Measures via Adapted Optimal Transport: Stability and Rates of Convergence

提出基于适应最优输运的收敛模式，恢复条件分布的弱连续性，并导出依赖度量插件估计器的O(N^{-1/3})收敛速率。

Convergence rate of estimators of clustered panel models with misclassification

We study kmeans clustering estimation of panel data models with a latent group structure and $N$ units and $T$ time periods under long panel asymptotics. We show that the group-specific coefficients can be estimated at the parametric root $NT$ rate even if error variances diverge as $T \to \infty$ and some units are asymptotically misclassified. This limit case approximates empirically relevant settings and is not covered by existing asymptotic results.

Consistencies and rates of convergence of jump-penalized least squares estimators

We study the asymptotics for jump-penalized least squares regression aiming at approximating a regression function by piecewise constant functions. Besides conventional consistency and convergence rates of the estimates in $L^2([0,1))$ our results cover other metrics like Skorokhod metric on the space of càdlàg functions and uniform metrics on $C([0,1])$. We will show that these estimators are in an adaptive sense rate optimal over certain classes of "approximation spaces." Special cases are the class of functions of bounded variation (piecewise) Hölder continuous functions of order $0<α\le1$ …

将协方差核估计量的上确界范数收敛速率与弱收敛性转移到函数主成分

Transferring supremum-norm rates and weak convergence of covariance kernel estimators to functional principal components

利用L2扰动理论，将协方差核估计量的上确界范数收敛速率与弱收敛性转移到函数主成分，并在离散观测模型下获得最优收敛速率与渐近正态性。

带边界流形上随机样本的Hessian估计量的收敛性

Convergence of Hessian estimator from random samples on a manifold with boundary

本文证明了在温和条件下，从带边界流形随机样本局部拟合二次多项式估计Hessian算子的方法渐近收敛，且非均匀采样和曲率效应可忽略，并简化了理论分析。

多变量尺度混合均匀密度估计的收敛速度

Convergence rates for estimating multivariate scale mixtures of uniform densities

本文提出了一种多变量尺度混合均匀密度估计方法，证明其收敛速度为立方根，且避免了维度灾难，同时提供了计算算法和实证分析。

局部线性估计在一般回归函数中对依赖函数数据的强一致性

Strong consistency of the local linear estimator for a generalized regression function with dependent functional data

本文研究了在依赖函数数据下局部线性估计器的强一致性，通过模拟验证了其在预测准确性上的优势。

Current status data with competing risks: Consistency and rates of Convergence of the MLE

We study nonparametric estimation of the sub-distribution functions for current status data with competing risks. Our main interest is in the nonparametric maximum likelihood estimator (MLE), and for comparison we also consider a simpler ``naive estimator.'' Both types of estimators were studied by Jewell, van der Laan and Henneman [Biometrika (2003) 90 183--197], but little was known about their large sample properties. We have started to fill this gap, by proving that the estimators are consistent and converge globally and locally at rate $n^{1/3}$. We also show that this local rate of conve…

Minimax Optimal Convergence Rates for Estimating Ground Truth from Crowdsourced Labels

Crowdsourcing has become a primary means for label collection in many real-world machine learning applications. A classical method for inferring the true labels from the noisy labels provided by crowdsourcing workers is Dawid-Skene estimator. In this paper, we prove convergence rates of a projected EM algorithm for the Dawid-Skene estimator. The revealed exponent in the rate of convergence is shown to be optimal via a lower bound argument. Our work resolves the long standing issue of whether Dawid-Skene estimator has sound theoretical guarantees besides its good performance observed in practic…

Uniform Function Estimators in Reproducing Kernel Hilbert Spaces

This paper addresses the problem of regression to reconstruct functions, which are observed with superimposed errors at random locations. We address the problem in reproducing kernel Hilbert spaces. It is demonstrated that the estimator, which is often derived by employing Gaussian random fields, converges in the mean norm of the reproducing kernel Hilbert space to the conditional expectation and this implies local and uniform convergence of this function estimator. By preselecting the kernel, the problem does not suffer from the curse of dimensionality. The paper analyzes the statistical prop…

局部依赖随机图模型估计量的收敛速度与正态逼近

Rates of convergence and normal approximations for estimators of local dependence random graph models

本文首次为局部依赖随机图模型参数向量建立了估计与推断理论，基于单次图观测推导了最大似然估计的$\ell_2$误差非渐近界及极小极大最优收敛速度，并给出了多元正态逼近误差的非渐近界。

Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density

We present theoretical properties of the log-concave maximum likelihood estimator of a density based on an independent and identically distributed sample in $\mathbb{R}^d$. Our study covers both the case where the true underlying density is log-concave, and where this model is misspecified. We begin by showing that for a sequence of log-concave densities, convergence in distribution implies much stronger types of convergence -- in particular, it implies convergence in Hellinger distance and even in certain exponentially weighted total variation norms. In our main result, we prove the existence…

双不变测地线回归：存在性、唯一性与收敛性

Bi-invariant Geodesic Regression: Existence, Uniqueness, and Convergence

本文研究李群上双不变测地线回归估计量的局部存在唯一性，并证明所提迭代算法线性收敛，给出邻域显式界限。

Some upper bounds for the rate of convergence of penalized likelihood context tree estimators

We find upper bounds for the probability of underestimation and overestimation errors in penalized likelihood context tree estimation. The bounds are explicit and applies to processes of not necessarily finite memory. We allow for general penalizing terms and we give conditions over the maximal depth of the estimated trees in order to get strongly consistent estimates. This generalizes previous results obtained in the case of estimation of the order of a Markov chain.

Optimal rates for plug-in estimators of density level sets

In the context of density level set estimation, we study the convergence of general plug-in methods under two main assumptions on the density for a given level $λ$. More precisely, it is assumed that the density (i) is smooth in a neighborhood of $λ$ and (ii) has $γ$-exponent at level $λ$. Condition (i) ensures that the density can be estimated at a standard nonparametric rate and condition (ii) is similar to Tsybakov's margin assumption which is stated for the classification framework. Under these assumptions, we derive optimal rates of convergence for plug-in estimators. Explicit convergence…

Convergence rates for density estimators of weakly dependent time series

Assuming that $(X_t)_{t\in\Z}$ is a vector valued time series with a common marginal distribution admitting a density $f$, our aim is to provide a wide range of consistent estimators of $f$. We consider different methods of estimation of the density as kernel, projection or wavelets ones. Various cases of weakly dependent series are investigated including the Doukhan & Louhichi (1999)'s $η$-weak dependence condition, and the $\tilde ϕ$-dependence of Dedecker & Prieur (2005). We thus obtain results for Markov chains, dynamical systems, bilinear models, non causal Moving Average... From a moment…

High-dimensional genome-wide association study and misspecified mixed model analysis

We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent gnome-wide association studies. The asymptotic analysis establishes consistency of the REML estimator of the variance of the errors in the LMM, and convergence in probability of the REML estimator of the variance of the random effects in the LMM to a certain limit, which is equal to the true variance of the random effects multiplied by the limiting proportion of the nonzero random effects present in the LMM. The aymptotic results also …

Bayesian MISE convergence rates of Polya urn based density estimators: asymptotic comparisons and choice of prior parameters

Mixture models are well-known for their versatility, and the Bayesian paradigm is a suitable platform for mixture analysis, particularly when the number of components is unknown. Bhattacharya (2008) introduced a mixture model based on the Dirichlet process, where an upper bound on the unknown number of components is to be specified. Here we consider a Bayesian asymptotic framework for objectively specifying the upper bound, which we assume to depend on the sample size. In particular, we define a Bayesian analogue of the mean integrated squared error (Bayesian MISE), and select that form of the…

Parameter Identification with Finite-Convergence Time Alertness Preservation

In this brief note we present two new parameter identifiers whose estimates converge in finite time under weak interval excitation assumptions. The main novelty is that, in contrast with other finite-convergence time (FCT) estimators, our schemes preserve the FCT property when the parameters change. The previous versions of our FCT estimators can track the parameter variations only asymptotically. Continuous-time and discrete-time versions of the new estimators are presented

在区间激励假设下，标准 LS 参数估计器确保线性回归方程的有限收敛时间

Standard LSParameter Estimators Ensure Finite Convergence Time for Linear Regression Equations Under an Interval Excitation Assumption

本文阐明在区间激励条件下，标准最小二乘估计器能使线性回归方程参数在满足该条件所需时长内有限收敛。