We study non-parametric estimation of an unknown density with support in R (respectively R+). The proposed estimation procedure is based on the projection on finite dimensional subspaces spanned by the Hermite (respectively the Laguerre) functions. The focus of this paper is to introduce a data-driven aggregation approach in order to deal with the upcoming bias-variance trade-off. Our novel procedure integrates the usual model selection method as a limit case. We show the oracle- and the minimax-optimality of the data-driven aggregated density estimator and hence its adaptivity. We present results of a simulation study which allow to compare the finite sample performance of the data-driven estimators using model selection compared to the new aggregation.
In this paper we prove the optimality of an aggregation procedure. We prove lower bounds for aggregation of model selection type of $M$ density estimators for the Kullback-Leiber divergence (KL), the Hellinger's distance and the $L\_1$-distance. The lower bound, with respect to the KL distance, can be achieved by the on-line type estimate suggested, among others, by Yang (2000). Combining these results, we state that $\log M/n$ is an optimal rate of aggregation in the sense of Tsybakov (2003), where $n$ is the sample size.
We derive the optimal signed variable in general case kernels for the classical statistic density estimation, which are some generalization of the famous Epanechnikov's ones.
The aim of this paper is to define a nonlinear least squares estimator for the spectral parameters of a spherical autoregressive process of order 1 in a parametric setting. Furthermore, we investigate on its asymptotic properties, such as weak consistency and asymptotic normality.
We study the maximum likelihood estimator of density of $n$ independent observations, under the assumption that it is well approximated by a mixture with a large number of components. The main focus is on statistical properties with respect to the Kullback-Leibler loss. We establish risk bounds taking the form of sharp oracle inequalities both in deviation and in expectation. A simple consequence of these bounds is that the maximum likelihood estimator attains the optimal rate $((\log K)/n)^{1/2}$, up to a possible logarithmic correction, in the problem of convex aggregation when the number $K…
In a parametric framework, the paper is devoted to the study of a new estimation procedure for the inverse filter and the level noise in a complex noisy blind discrete deconvolution model. Our estimation method is a consequence of the sharp exploitation of the specifical properties of the Hankel forms. The distribution of the input signal is also estimated. The strong consistency and the asymptotic distribution of all estimates are established. A consistent simulation study is added in order to demonstrate empirically the computational performance of our estimation procedures.
We consider non-parametric estimation problems in the presence of dependent data, notably non-parametric regression with random design and non-parametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterized by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristi…
Literatures in state space models focus on parametric inference and prediction, which fail if the state space model is not fully specified and the maximum likelihood estimation does not work. In this paper, we assume the state transition matrix and the distribution of state noises are unknown. Under this assumption, we provide methods to consistently estimate these terms. In addition, we introduce an algorithm to construct consistent prediction intervals for state vectors and future observations. We complement the asymptotic results with several numerical experiments.
In the present work, we provide the asymptotic behavior of the residual-past entropy, of the mean residual-past lifetime distribution and of the residual-past inaccuracy measure. We are interested in these measures of uncertainty in the discrete case. Almost sure rates of convergence and asymptotic normality results are established. Our theoretical results are validated by simulations
We focus on the parametric estimation of the distribution of a Markov environment from the observation of a single trajectory of a one-dimensional nearest-neighbor path evolving in this random environment. In the ballistic case, as the length of the path increases, we prove consistency, asymptotic normality and efficiency of the maximum likelihood estimator. Our contribution is two-fold: we cast the problem into the one of parameter estimation in a hidden Markov model (HMM) and establish that the bivariate Markov chain underlying this HMM is positive Harris recurrent. We provide different exam…
We establish some new non-asymptotical lower bounds for deviation of regular unbiased estimation of unknown parameter from its true value in different norms, alike the classical Rao-Kramer's inequality. We show that if the new norm is weaker that ordinary Hilbertian norm, that the rate of convergence of arbitrary regular unbiased estimate does not exceed $ 1/\sqrt{n}, $ and if the new norm is stronger that one, the rate of convergence of the well-known Maximal Likelihood Estimate (MLE) is also equal to $ 1/\sqrt{n}.
In the context of kernel density estimation, we give a characterization of the kernels for which the parametric mean integrated squared error rate $n^{-1}$ may be obtained, where $n$ is the sample size. Also, for the cases where this rate is attainable, we give an asymptotic bandwidth choice that makes the kernel estimator consistent in mean integrated squared error at that rate and a numerical example showing the superior performance of the superkernel estimator when the bandwidth is properly chosen.
The main purpose of this chapter is to present some theoretical aspects of parametric estimation of Lévy processes based on high-frequency sampling, with a focus on infinite activity pure-jump models. Asymptotics for several classes of explicit estimating functions are discussed. In addition to the asymptotic normality at several rates of convergence, a uniform tail-probability estimate for statistical random fields is given. As specific cases, we discuss method of moments for the stable Lévy processes in much greater detail, with briefly mentioning locally stable Lévy processes too. Also disc…
We study the problem of model selection type aggregation with respect to the Kullback-Leibler divergence for various probabilistic models. Rather than considering a convex combination of the initial estimators $f_1, \ldots, f_N$, our aggregation procedures rely on the convex combination of the logarithms of these functions. The first method is designed for probability density estimation as it gives an aggregate estimator that is also a proper density function, whereas the second method concerns spectral density estimation and has no such mass-conserving feature. We select the aggregation weigh…
Extending the ideas of [7], this paper aims at providing a kernel based non-parametric estimation of a new class of time varying AR(1) processes (Xt), with local stationarity and periodic features (with a known period T), inducing the definition Xt = at(t/nT)X t--1 + $ξ$t for t $\in$ N and with a t+T $\not\equiv$ at. Central limit theorems are established for kernel estima-tors as(u) reaching classical minimax rates and only requiring low order moment conditions of the white noise ($ξ$t)t up to the second order.
This paper deals with the parametric inference for integrated signals embedded in an additive Gaussian noise and observed at deterministic discrete instants which are not necessarily equidistant. The unknown parameter is multidimensional and compounded of a signal-of-interest parameter and a variance parameter of the noise. We state the consistency and the minimax efficiency of the maximum likelihood estimator and of the Bayesian estimator when the time of observation tends to $\infty$ and the delays between two consecutive observations tend to 0 or are only bounded. The class of signals in co…
This article introduces a non parametric warping model for functional data. When the outcome of an experiment is a sample of curves, data can be seen as realizations of a stochastic process, which takes into account the small variations between the different observed curves. The aim of this work is to define a mean pattern which represents the main behaviour of the set of all the realizations. So we define the structural expectation of the underlying stochastic function. Then we provide empirical estimators of this structural expectation and of each individual warping function. Consistency and…
We introduce estimation and test procedures through divergence optimization for discrete or continuous parametric models. This approach is based on a new dual representation for divergences. We treat point estimation and tests for simple and composite hypotheses, extending maximum likelihood technique. An other view at the maximum likelihood approach, for estimation and test, is given. We prove existence and consistency of the proposed estimates. The limit laws of the estimates and test statistics (including the generalized likelihood ratio one) are given both under the null and the alternativ…
We consider a recurrent Markov process which is an Itô semi-martingale. The Lévy kernel describes the law of its jumps. Based on observations X(0),X(Δ),...,X(nΔ), we construct an estimator for the Lévy kernel's density. We prove its consistency (as nΔ->\infty and Δ->0) and a central limit theorem. In the positive recurrent case, our estimator is asymptotically normal; in the null recurrent case, it is asymptotically mixed normal. Our estimator's rate of convergence equals the non-parametric minimax rate of smooth density estimation. The asymptotic bias and variance are analogous to those of th…
We consider the problem of estimating the predictive density of future observations from a non-parametric regression model. The density estimators are evaluated under Kullback--Leibler divergence and our focus is on establishing the exact asymptotics of minimax risk in the case of Gaussian errors. We derive the convergence rate and constant for minimax risk among Bayesian predictive densities under Gaussian priors and we show that this minimax risk is asymptotically equivalent to that among all density estimators.
This paper introduces a family of recursively defined estimators of the parameters of a diffusion process. We use ideas of stochastic algorithms for the construction of the estimators. Asymptotic consistency of these estimators and asymptotic normality of an appropriate normalization are proved. The results are applied to two examples from the financial literature; viz., Cox-Ingersoll-Ross' model and the constant elasticity of variance (CEV) process illustrate the use of the technique proposed herein.
We study minimum contrast estimation for parametric stationary determi-nantal point processes. These processes form a useful class of models for repulsive (or regular, or inhibitive) point patterns and are already applied in numerous statistical applications. Our main focus is on minimum contrast methods based on the Ripley's K-function or on the pair correlation function. Strong consistency and asymptotic normality of theses procedures are proved under general conditions that only concern the existence of the process and its regularity with respect to the parameters. A key ingredient of the p…
In this paper, we give a general time-varying parameter model, where the multidimensional parameter possibly includes jumps. The quantity of interest is defined as the integrated value over time of the parameter process $Θ= T^{-1} \int_0^T θ_t^* dt$. We provide a local parametric estimator (LPE) of $Θ$ and conditions under which we can show the central limit theorem. Roughly speaking those conditions correspond to some uniform limit theory in the parametric version of the problem. The framework is restricted to the specific convergence rate $n^{1/2}$. Several examples of LPE are studied: estim…
This article studies a trimmed version of the Nadaraya-Watson estimator to estimate the unknown non-parametric regression function. The characterization of the estimator through minimization problem is established, and its pointwise asymptotic distribution is also derived. The robustness property of the proposed estimator is also studied through breakdown point. Besides, the asymptotic efficiency study along with an extensive simulation study shows that this estimator performs well for various cases. The practicability of the estimator is shown for three benchmark real data as well.
Indirect inference estimators (i.e., simulation-based minimum distance estimators) in a parametric model that are based on auxiliary non-parametric maximum likelihood density estimators are shown to be asymptotically normal. If the parametric model is correctly specified, it is furthermore shown that the asymptotic variance-covariance matrix equals the inverse of the Fisher-information matrix. These results are based on uniform-in-parameters convergence rates and a uniform-in-parameters Donsker-type theorem for non-parametric maximum likelihood density estimators.
In this paper, different strands of literature are combined in order to obtain algorithms for semi-parametric estimation of discrete choice models that include the modelling of unobserved heterogeneity by using mixing distributions for the parameters defining the preferences. The models use the theory on non-parametric maximum likelihood estimation (NP-MLE) that has been developed for general mixing models. The expectation-maximization (EM) techniques used in the NP-MLE literature are combined with strategies for choosing appropriate approximating models using adaptive grid techniques. \\ Join…
In this paper, we propose an estimator of Foster, Greer and Thorbecke class of measures $\displaystyle P(z,α) = \int_0^{z}\Big(\frac{z-x}{z}\Big)^αf(x)\, dx$, where $z>0$ is the poverty line, $f$ is the probabily density function of the income distribution and $α$ is the so-called poverty aversion. The estimator is constructed with a bias reduced kernel estimator. Uniform almost sure consistency and uniform mean square consistenty are established. A simulation study indicates that our new estimator performs well.
Stanford University（斯坦福大学）；University of Cyprus（塞浦路斯大学）
We consider the parameter estimation problem of a probabilistic generative model prescribed using a natural exponential family of distributions. For this problem, the typical maximum likelihood estimator usually overfits under limited training sample size, is sensitive to noise and may perform poorly on downstream predictive tasks. To mitigate these issues, we propose a distributionally robust maximum likelihood estimator that minimizes the worst-case expected log-loss uniformly over a parametric Kullback-Leibler ball around a parametric nominal distribution. Leveraging the analytical expressi…
We show that the jumps correlation matrix of a multivariate Hawkes process is related to the Hawkes kernel matrix through a system of Wiener-Hopf integral equations. A Wiener-Hopf argument allows one to prove that this system (in which the kernel matrix is the unknown) possesses a unique causal solution and consequently that the second-order properties fully characterize a Hawkes process. The numerical inversion of this system of integral equations allows us to propose a fast and efficient method, which main principles were initially sketched in [Bacry and Muzy, 2013], to perform a non-paramet…
In an earthquake event, the combination of a strong mainshock and damaging aftershocks is often the cause of severe structural damages and/or high death tolls. The objective of this paper is to provide estimation for the probability of such extreme events where the mainshock and the largest aftershocks exceed certain thresholds. Two approaches are illustrated and compared -- a parametric approach based on previously observed stochastic laws in earthquake data, and a non-parametric approach based on bivariate extreme value theory. We analyze the earthquake data from the North Anatolian Fault Zo…
For a class of parametric modal regression models with measurement error, a simulation extrapolation estimation procedure is proposed in this paper for estimating the modal regression coefficients. Large sample properties of the proposed estimation procedure, including the consistency and asymptotic normality, are thoroughly investigated. Simulation studies are conducted to evaluate its robustness to potential outliers and the effectiveness in reducing the bias caused by the measurement error.
In multiple testing, the family-wise error rate can be bounded under some conditions by the copula of the test statistics. Assuming that this copula is Archimedean, we consider two non-parametric Archimedean generator estimators. More specifically, we use the non-parametric estimator from Genest et al. (2011) and a slight modification thereof. In simulations, we compare the resulting multiple tests with the Bonferroni test and the multiple test derived from the true generator as baselines.
In this paper, we perform registration of noisy curves. We provide an appropriate model in estimating the rotation and scaling parameters to adjust a set of curves through a M-estimation procedure. We prove the consistency and the asymptotic normality of our estimators. Numerical simulation and a real life aeronautic example are given to illustrate our methodology.
For the general parametric regression models with covariates contaminated with normal measurement errors, this paper proposes an accelerated version of the classical simulation extrapolation algorithm to estimate the unknown parameters in the regression function. By applying the conditional expectation directly to the target function, the proposed algorithm successfully removes the simulation step, by generating an estimation equation either for immediate use or for extrapolating, thus significantly reducing the computational time. Large sample properties of the resulting estimator, including …
A Copula density estimation method that is based on a finite mixture of heterogeneous parametric copula densities is proposed here. More specifically, the mixture components are Clayton, Frank, Gumbel, T, and normal copula densities, which are capable of capturing lower tail,strong central, upper tail, heavy tail, and symmetrical elliptical dependence, respectively. The model parameters are estimated by an interior-point algorithm for the constrained maximum likelihood problem. The interior-point algorithm is compared with the commonly used EM algorithm. Simulation and real data application sh…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon Rényi-Tsallis entropies, and the mutual information estimates of a pair of discrete random variables. Almost sure consistency and central limit Theorems are established. Our theorical results are validated by simulations.
Product models of low dimensional experts are a powerful way to avoid the curse of dimensionality. We present the ``under-complete product of experts' (UPoE), where each expert models a one dimensional projection of the data. The UPoE is fully tractable and may be interpreted as a parametric probabilistic model for projection pursuit. Its ML learning rules are identical to the approximate learning rules proposed before for under-complete ICA. We also derive an efficient sequential learning algorithm and discuss its relationship to projection pursuit density estimation and feature induction alg…
There is a long-standing debate in the statistical, epidemiological and econometric fields as to whether nonparametric estimation that uses data-adaptive methods, like machine learning algorithms in model fitting, confer any meaningful advantage over simpler, parametric approaches in real-world, finite sample estimation of causal effects. We address the question: when trying to estimate the effect of a treatment on an outcome, across a universe of reasonable data distributions, how much does the choice of nonparametric vs.~parametric estimation matter? Instead of answering this question with s…
The Fisher information matrix (FIM) is a foundational concept in statistical signal processing. The FIM depends on the probability distribution, assumed to belong to a smooth parametric family. Traditional approaches to estimating the FIM require estimating the probability distribution function (PDF), or its parameters, along with its gradient or Hessian. However, in many practical situations the PDF of the data is not known but the statistician has access to an observation sample for any parameter value. Here we propose a method of estimating the FIM directly from sampled data that does not r…
Multi-dimensional Hawkes process (MHP) is a class of self and mutually exciting point processes that find wide range of applications -- from prediction of earthquakes to modelling of order books in high frequency trading. This paper makes two major contributions, we first find an unbiased estimator for the log-likelihood estimator of the Hawkes process to enable efficient use of the stochastic gradient descent method for maximum likelihood estimation. The second contribution is, we propose a specific single hidden layered neural network for the non-parametric estimation of the underlying kerne…
Density Estimation Trees (DETs) are decision trees trained on a multivariate dataset to estimate its probability density function. While not competitive with kernel techniques in terms of accuracy, they are incredibly fast, embarrassingly parallel and relatively small when stored to disk. These properties make DETs appealing in the resource-expensive horizon of the LHC data analysis. Possible applications may include selection optimization, fast simulation and fast detector calibration. In this contribution I describe the algorithm, made available to the HEP community in a RooFit implementatio…
We study a new parametric approach for particular hidden stochastic models such as the Stochastic Volatility model. This method is based on contrast minimization and deconvolution. After proving consistency and asymptotic normality of the estimation leading to asymptotic confidence intervals, we provide a thorough numerical study, which compares most of the classical methods that are used in practice (Quasi Maximum Likelihood estimator, Simulated Expectation Maximization Likelihood estimator and Bayesian estimators). We prove that our estimator clearly outperforms the Maximum Likelihood Estima…
The problem of estimating the baseline signal from multisample noisy curves is investigated. We consider the functional mixed effects model, and we suppose that the functional fixed effect belongs to the Besov class. This framework allows us to model curves that can exhibit strong irregularities, such as peaks or jumps for instance. The lower bound for the $L_2$ minimax risk is provided, as well as the upper bound of the minimax rate, that is derived by constructing a wavelet estimator for the functional fixed effect. Our work constitutes the first theoretical functional results in multisample…
We consider the semi-parametric estimation of a scale parameter of a one-dimensional Gaussian process with known smoothness. We suggest an estimator based on quadratic variations and on the moment method. We provide asymptotic approximations of the mean and variance of this estimator, together with asymptotic normality results, for a large class of Gaussian processes. We allow for general mean functions and study the aggregation of several estimators based on various variation sequences. In extensive simulation studies, we show that the asymptotic results accurately depict thefinite-sample sit…
Adaptive smoothness of function estimation in the three classical problems of the non-parametrical statistic in the three classical problems of the non-parametrical statistic
We derive the posteror contraction rate for non-parametric Bayesian estimation of a deterministic dispersion coefficient of a linear stochastic differential equation.
This paper considers the problem of estimating a periodic function in a continuous time regression model with a general square integrable semimartingale noise. A model selection adaptive procedure is proposed. Sharp non-asymptotic oracle inequalities have been derived.
We consider parametric estimation for a parabolic linear second order stochastic partial differential equation (SPDE) from high frequency data which are observed in time and space. By using thinned data obtained from the high frequency data, adaptive estimators of the coefficient parameters including the volatility parameter of a parabolic linear SPDE model are proposed. Moreover, we give some examples and simulation results of the adaptive estimators of the SPDE model based on the high frequency data.
We consider non-parametric Bayesian estimation of the drift coefficient of a one-dimensional stochastic differential equation from discrete-time observations on the solution of this equation. Under suitable regularity conditions that are weaker than those previosly suggested in the literature, we establish posterior consistency in this context. Furthermore, we show that posterior consistency extends to the multidimensional setting as well, which, to the best of our knowledge, is a new result in this setting.
We consider a combined state and drift estimation problem for the linear stochastic heat equation. The infinite-dimensional Bayesian inference problem is formulated in terms of the Kalman-Bucy filter over an extended state space, and its long-time asymptotic properties are studied. Asymptotic posterior contraction rates in the unknown drift function are the main contribution of this paper. Such rates have been studied before for stationary non-parametric Bayesian inverse problems, and here we demonstrate the consistency of our time-dependent formulation with these previous results building upo…
The paper is devoted to the study of a parametric deformation model of independent and identically random variables. Firstly, we construct an efficient and very easy to compute recursive estimate of the parameter. Our stochastic estimator is similar to the Robbins-Monro procedure where the contrast function is the Wasserstein distance. Secondly, we propose a recursive estimator similar to that of Parzen-Rosenblatt kernel density estimator in order to estimate the density of the random variables. This estimate takes into account the previous estimation of the parameter of the model. Finally, we…
In this article, we consider a jump diffusion process (X_t)observed at discrete times t=0,Delta,...,nDelta. The sampling interval Delta tends to 0 and nDelta tends to infinity. We assume that (X_t) is ergodic, strictly stationary and exponentially β-mixing. We use a penalized least-square approach to compute two adaptive estimators of the drift function b. We provide bounds for the risks of the two estimators.
In this paper,we consider a macro approximation of the flow of a risk reserve, The process is observed at discrete time points. Because we cannot directly observe each jump time and size then we will make use of a technique for identifying the times when jumps larger than a suitably defined threshold occurred. We estimate the jump size and survival probability of our risk process from discrete observations.
