The paper aims at reconsidering the famous Le Cam LAN theory. The main features of the approach which make it different from the classical one are as follows: (1) the study is nonasymptotic, that is, the sample size is fixed and does not tend to infinity; (2) the parametric assumption is possibly misspecified and the underlying data distribution can lie beyond the given parametric family. These two features enable to bridge the gap between parametric and nonparametric theory and to build a unified framework for statistical estimation. The main results include large deviation bounds for the (quasi) maximum likelihood and the local quadratic bracketing of the log-likelihood process. The latter yields a number of important corollaries for statistical inference: concentration, confidence and risk bounds, expansion of the maximum likelihood estimate, etc. All these corollaries are stated in a nonclassical way admitting a model misspecification and finite samples. However, the classical asymptotic results including the efficiency bounds can be easily derived as corollaries of the obtained nonasymptotic statements. At the same time, the new bracketing device works well in the situations with large or growing parameter dimension in which the classical parametric theory fails. The general results are illustrated for the i.i.d. setup as well as for generalized linear and median estimation. The results apply for any dimension of the parameter space and provide a quantitative lower bound on the sample size yielding the root-n accuracy.
We derive the optimal signed variable in general case kernels for the classical statistic density estimation, which are some generalization of the famous Epanechnikov's ones.
The aim of this paper is to define a nonlinear least squares estimator for the spectral parameters of a spherical autoregressive process of order 1 in a parametric setting. Furthermore, we investigate on its asymptotic properties, such as weak consistency and asymptotic normality.
Literatures in state space models focus on parametric inference and prediction, which fail if the state space model is not fully specified and the maximum likelihood estimation does not work. In this paper, we assume the state transition matrix and the distribution of state noises are unknown. Under this assumption, we provide methods to consistently estimate these terms. In addition, we introduce an algorithm to construct consistent prediction intervals for state vectors and future observations. We complement the asymptotic results with several numerical experiments.
In the present work, we provide the asymptotic behavior of the residual-past entropy, of the mean residual-past lifetime distribution and of the residual-past inaccuracy measure. We are interested in these measures of uncertainty in the discrete case. Almost sure rates of convergence and asymptotic normality results are established. Our theoretical results are validated by simulations
We study non-parametric estimation of an unknown density with support in R (respectively R+). The proposed estimation procedure is based on the projection on finite dimensional subspaces spanned by the Hermite (respectively the Laguerre) functions. The focus of this paper is to introduce a data-driven aggregation approach in order to deal with the upcoming bias-variance trade-off. Our novel procedure integrates the usual model selection method as a limit case. We show the oracle- and the minimax-optimality of the data-driven aggregated density estimator and hence its adaptivity. We present res…
Extending the ideas of [7], this paper aims at providing a kernel based non-parametric estimation of a new class of time varying AR(1) processes (Xt), with local stationarity and periodic features (with a known period T), inducing the definition Xt = at(t/nT)X t--1 + $ξ$t for t $\in$ N and with a t+T $\not\equiv$ at. Central limit theorems are established for kernel estima-tors as(u) reaching classical minimax rates and only requiring low order moment conditions of the white noise ($ξ$t)t up to the second order.
This paper deals with the parametric inference for integrated signals embedded in an additive Gaussian noise and observed at deterministic discrete instants which are not necessarily equidistant. The unknown parameter is multidimensional and compounded of a signal-of-interest parameter and a variance parameter of the noise. We state the consistency and the minimax efficiency of the maximum likelihood estimator and of the Bayesian estimator when the time of observation tends to $\infty$ and the delays between two consecutive observations tend to 0 or are only bounded. The class of signals in co…
In this paper, we give a general time-varying parameter model, where the multidimensional parameter possibly includes jumps. The quantity of interest is defined as the integrated value over time of the parameter process $Θ= T^{-1} \int_0^T θ_t^* dt$. We provide a local parametric estimator (LPE) of $Θ$ and conditions under which we can show the central limit theorem. Roughly speaking those conditions correspond to some uniform limit theory in the parametric version of the problem. The framework is restricted to the specific convergence rate $n^{1/2}$. Several examples of LPE are studied: estim…
This article studies a trimmed version of the Nadaraya-Watson estimator to estimate the unknown non-parametric regression function. The characterization of the estimator through minimization problem is established, and its pointwise asymptotic distribution is also derived. The robustness property of the proposed estimator is also studied through breakdown point. Besides, the asymptotic efficiency study along with an extensive simulation study shows that this estimator performs well for various cases. The practicability of the estimator is shown for three benchmark real data as well.
In this paper, different strands of literature are combined in order to obtain algorithms for semi-parametric estimation of discrete choice models that include the modelling of unobserved heterogeneity by using mixing distributions for the parameters defining the preferences. The models use the theory on non-parametric maximum likelihood estimation (NP-MLE) that has been developed for general mixing models. The expectation-maximization (EM) techniques used in the NP-MLE literature are combined with strategies for choosing appropriate approximating models using adaptive grid techniques. \\ Join…
In this paper, we propose an estimator of Foster, Greer and Thorbecke class of measures $\displaystyle P(z,α) = \int_0^{z}\Big(\frac{z-x}{z}\Big)^αf(x)\, dx$, where $z>0$ is the poverty line, $f$ is the probabily density function of the income distribution and $α$ is the so-called poverty aversion. The estimator is constructed with a bias reduced kernel estimator. Uniform almost sure consistency and uniform mean square consistenty are established. A simulation study indicates that our new estimator performs well.
Stanford University（斯坦福大学）；University of Cyprus（塞浦路斯大学）
We consider the parameter estimation problem of a probabilistic generative model prescribed using a natural exponential family of distributions. For this problem, the typical maximum likelihood estimator usually overfits under limited training sample size, is sensitive to noise and may perform poorly on downstream predictive tasks. To mitigate these issues, we propose a distributionally robust maximum likelihood estimator that minimizes the worst-case expected log-loss uniformly over a parametric Kullback-Leibler ball around a parametric nominal distribution. Leveraging the analytical expressi…
An important research topic in survival analysis is related to the modeling and estimation of the cure rate, i.e. the proportion of subjects that will never experience the event of interest. However, most estimation methods proposed so far in the literature do not handle the case of insufficient follow-up, that is when the right end point of the support of the censoring time is strictly less than that of the survival time of the susceptible subjects, and consequently these estimators overestimate the cure rate in that case. We fill this gap by proposing a new estimator of the cure rate that ma…
In an earthquake event, the combination of a strong mainshock and damaging aftershocks is often the cause of severe structural damages and/or high death tolls. The objective of this paper is to provide estimation for the probability of such extreme events where the mainshock and the largest aftershocks exceed certain thresholds. Two approaches are illustrated and compared -- a parametric approach based on previously observed stochastic laws in earthquake data, and a non-parametric approach based on bivariate extreme value theory. We analyze the earthquake data from the North Anatolian Fault Zo…
For a class of parametric modal regression models with measurement error, a simulation extrapolation estimation procedure is proposed in this paper for estimating the modal regression coefficients. Large sample properties of the proposed estimation procedure, including the consistency and asymptotic normality, are thoroughly investigated. Simulation studies are conducted to evaluate its robustness to potential outliers and the effectiveness in reducing the bias caused by the measurement error.
In multiple testing, the family-wise error rate can be bounded under some conditions by the copula of the test statistics. Assuming that this copula is Archimedean, we consider two non-parametric Archimedean generator estimators. More specifically, we use the non-parametric estimator from Genest et al. (2011) and a slight modification thereof. In simulations, we compare the resulting multiple tests with the Bonferroni test and the multiple test derived from the true generator as baselines.
In this paper, we perform registration of noisy curves. We provide an appropriate model in estimating the rotation and scaling parameters to adjust a set of curves through a M-estimation procedure. We prove the consistency and the asymptotic normality of our estimators. Numerical simulation and a real life aeronautic example are given to illustrate our methodology.
A Robbins-Monro algorithm for non-parametric estimation of NAR process with Markov-Switching: asymptotic normality
Aix-Marseille University（艾克斯-马赛大学）；Escuela de Matemáticas, Facultad de Ciencias, Universidad Central de Venezuela（委内瑞拉中央大学理学院数学系）；Departamento de Matemáticas, FACYT, Universidad de Carabobo（卡拉沃博大学FACYT数学系）
For the general parametric regression models with covariates contaminated with normal measurement errors, this paper proposes an accelerated version of the classical simulation extrapolation algorithm to estimate the unknown parameters in the regression function. By applying the conditional expectation directly to the target function, the proposed algorithm successfully removes the simulation step, by generating an estimation equation either for immediate use or for extrapolating, thus significantly reducing the computational time. Large sample properties of the resulting estimator, including …
A Copula density estimation method that is based on a finite mixture of heterogeneous parametric copula densities is proposed here. More specifically, the mixture components are Clayton, Frank, Gumbel, T, and normal copula densities, which are capable of capturing lower tail,strong central, upper tail, heavy tail, and symmetrical elliptical dependence, respectively. The model parameters are estimated by an interior-point algorithm for the constrained maximum likelihood problem. The interior-point algorithm is compared with the commonly used EM algorithm. Simulation and real data application sh…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon Rényi-Tsallis entropies, and the mutual information estimates of a pair of discrete random variables. Almost sure consistency and central limit Theorems are established. Our theorical results are validated by simulations.
There is a long-standing debate in the statistical, epidemiological and econometric fields as to whether nonparametric estimation that uses data-adaptive methods, like machine learning algorithms in model fitting, confer any meaningful advantage over simpler, parametric approaches in real-world, finite sample estimation of causal effects. We address the question: when trying to estimate the effect of a treatment on an outcome, across a universe of reasonable data distributions, how much does the choice of nonparametric vs.~parametric estimation matter? Instead of answering this question with s…
Multi-dimensional Hawkes process (MHP) is a class of self and mutually exciting point processes that find wide range of applications -- from prediction of earthquakes to modelling of order books in high frequency trading. This paper makes two major contributions, we first find an unbiased estimator for the log-likelihood estimator of the Hawkes process to enable efficient use of the stochastic gradient descent method for maximum likelihood estimation. The second contribution is, we propose a specific single hidden layered neural network for the non-parametric estimation of the underlying kerne…
Texas A&M University（德州农工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Northeastern University（东北大学）
Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization. The main bottleneck in high-dimensional density estimation is the prohibitive computational cost and the slow convergence rate. In this paper, we propose novel estimators for high-dimensional non-parametric density estimation called the adaptive hyperbolic cross density estimators, which enjoys nice convergence properties in the mixed smooth Sobolev spaces. As modifications of the usual Sobolev spaces, the mixed smooth Sobolev spaces are more suitable for describing high-dimensional…
Institute of Statistical Science, Academia Sinica（中央研究院统计科学研究所）
The model interpretation is essential in many application scenarios and to build a classification model with a ease of model interpretation may provide useful information for further studies and improvement. It is common to encounter with a lengthy set of variables in modern data analysis, especially when data are collected in some automatic ways. This kinds of datasets may not collected with a specific analysis target and usually contains redundant features, which have no contribution to a the current analysis task of interest. Variable selection is a common way to increase the ability of mod…
Centre Borelli, ENS Paris-Saclay, Université Paris-Saclay（巴黎萨克雷大学巴黎萨克雷高等师范学院博雷利中心）
Consider each node of a graph to be generating a data stream that is synchronized and observed at near real-time. At a change-point $τ$, a change occurs at a subset of nodes $C$, which affects the probability distribution of their associated node streams. In this paper, we propose a novel kernel-based method to both detect $τ$ and localize $C$, based on the direct estimation of the likelihood-ratio between the post-change and the pre-change distributions of the node streams. Our main working hypothesis is the smoothness of the likelihood-ratio estimates over the graph, i.e connected nodes are …
Adaptive smoothness of function estimation in the three classical problems of the non-parametrical statistic in the three classical problems of the non-parametrical statistic
