We consider non-parametric estimation problems in the presence of dependent data, notably non-parametric regression with random design and non-parametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterized by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristics of the function of interest, which are not known in practice. The main issue addressed in our work is an adaptive choice of this dimension parameter combining model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski (2011). We show that this data-driven estimator can attain the lower risk bound up to a constant provided a fast decay of the mixing coefficients.
Estimation of extreme value copulas is often required in situations where available data are sparse. Parametric methods may then be the preferred approach. A possible way of defining parametric families that are simple and, at the same time, cover a large variety of multivariate extremal dependence structures is to build models based on spectral measures. This approach is considered here. Parametric families of spectral measures are defined as convex hulls of suitable basis elements, and parameters are estimated by projecting an initial nonparametric estimator on these finite-dimensional space…
We consider the estimation of a structural function which models a non-parametric relationship between a response and an endogenous regressor given an instrument in presence of dependence in the data generating process. Assuming an independent and identically distributed (iid.) sample it has been shown in Johannes and Schwarz (2010) that a least squares estimator based on dimension reduction and thresholding can attain minimax-optimal rates of convergence up to a constant. As this estimation procedure requires an optimal choice of a dimension parameter with regard amongst others to certain cha…
We consider parametric estimation and tests for multi-dimensional diffusion processes with a small dispersion parameter $\varepsilon$ from discrete observations. For parametric estimation of diffusion processes, the main target is to estimate the drift parameter and the diffusion parameter. In this paper, we propose two types of adaptive estimators for both parameters and show their asymptotic properties under $\varepsilon\to0$, $n\to\infty$ and the balance condition that $(\varepsilon n^ρ)^{-1} =O(1)$ for some $ρ>0$. Using these adaptive estimators, we also introduce consistent adaptive testi…
In the world of multivariate extremes, estimation of the dependence structure still presents a challenge and an interesting problem. A procedure for the bivariate case is presented that opens the road to a similar way of handling the problem in a truly multivariate setting. We consider a semi-parametric model in which the stable tail dependence function is parametrically modeled. Given a random sample from a bivariate distribution function, the problem is to estimate the unknown parameter. A method of moments estimator is proposed where a certain integral of a nonparametric, rank-based estimat…
We propose a new fully non-parametric two-step adaptive bandwidth selection method for kernel estimators of spatial point process intensity functions based on the Campbell-Mecke formula and Abramson's square root law. We present a simulation study to assess its performance relative to the Cronie-Van Lieshout global bandwidth selector and apply the technique to data on induced earthquakes in theGroningen gas field.
In the context of multiple hypotheses testing, the proportion $π_0$ of true null hypotheses in the pool of hypotheses to test often plays a crucial role, although it is generally unknown a priori. A testing procedure using an implicit or explicit estimate of this quantity in order to improve its efficency is called adaptive. In this paper, we focus on the issue of False Discovery Rate (FDR) control and we present new adaptive multiple testing procedures with control of the FDR. First, in the context of assuming independent $p$-values, we present two new procedures and give a unified review of …
A class of estimating functions is introduced for the regression parameter of the Cox proportional hazards model to allow unknown failure statuses on some study subjects. The consistency and asymptotic normality of the resulting estimators are established under mild conditions. An adaptive estimator which achieves the minimum variance-covariance bound of the class is constructed. Numerical studies demonstrate that the asymptotic approximations are adequate for practical use and that the efficiency gain of the adaptive estimator over the complete-case analysis can be quite substantial. Similar …
We study a non-parametric approach to multivariate density estimation. The estimators are piecewise constant density functions supported by binary partitions. The partition of the sample space is learned by maximizing the likelihood of the corresponding histogram on that partition. We analyze the convergence rate of the sieve maximum likelihood estimator, and reach a conclusion that for a relatively rich class of density functions the rate does not directly depend on the dimension. This suggests that, under certain conditions, this method is immune to the curse of dimensionality, in the sense …
Multivariate extreme value theory is concerned with modeling the joint tail behavior of several random variables. Existing work mostly focuses on asymptotic dependence, where the probability of observing a large value in one of the variables is of the same order as observing a large value in all variables simultaneously. However, there is growing evidence that asymptotic independence is equally important in real world applications. Available statistical methodology in the latter setting is scarce and not well understood theoretically. We revisit non-parametric estimation and introduce rank-bas…
We propose a series-based nonparametric specification test for a regression function when data are spatially dependent, the `space' being of a general economic or social nature. Dependence can be parametric, parametric with increasing dimension, semiparametric or any combination thereof, thus covering a vast variety of settings. These include spatial error models of varying types and levels of complexity. Under a new smooth spatial dependence condition, our test statistic is asymptotically standard normal. To prove the latter property, we establish a central limit theorem for quadratic forms i…
Accurate statistical inference in logistic regression models remains a critical challenge when the ratio between the number of parameters and sample size is not negligible. This is because approximations based on either classical asymptotic theory or bootstrap calculations are grossly off the mark. This paper introduces a resized bootstrap method to infer model parameters in arbitrary dimensions. As in the parametric bootstrap, we resample observations from a distribution, which depends on an estimated regression coefficient sequence. The novelty is that this estimate is actually far from the …
We introduce a balloon estimator in a generalized expectation-maximization method for estimating all parameters of a Gaussian mixture model given one data sample per mixture component. Instead of limiting explicitly the model size, this regularization strategy yields low-complexity sparse models where the number of effective mixture components reduces with an increase of a smoothing probability parameter $\mathbf{P>0}$. This semi-parametric method bridges from non-parametric adaptive kernel density estimation (KDE) to parametric ordinary least-squares when $\mathbf{P=1}$. Experiments show that…
Adaptive smoothness of function estimation in the three classical problems of the non-parametrical statistic in the three classical problems of the non-parametrical statistic
We consider the framework of non-stationary stochastic optimization [Besbes et al, 2015] with squared error losses and noisy gradient feedback where the dynamic regret of an online learner against a time varying comparator sequence is studied. Motivated from the theory of non-parametric regression, we introduce a new variational constraint that enforces the comparator sequence to belong to a discrete $k^{th}$ order Total Variation ball of radius $C_n$. This variational constraint models comparators that have piece-wise polynomial structure which has many relevant practical applications [Tibshi…
Often in multiple testing, the hypotheses appear in non-overlapping blocks with the associated $p$-values exhibiting dependence within but not between blocks. We consider adapting the Benjamini-Hochberg method for controlling the false discovery rate (FDR) and the Bonferroni method for controlling the familywise error rate (FWER) to such dependence structure without losing their ultimate controls over the FDR and FWER, respectively, in a non-asymptotic setting. We present variants of conventional adaptive Benjamini-Hochberg and Bonferroni methods with proofs of their respective controls over t…
This paper considers the regional estimation of high quantiles of annual maximal river flow distributions $F$, an important problem from flood frequency analysis. Even though this particular problem has been addressed by many papers, less attention has been payed to incorporating seasonal variability and spatial dependence into the methods. We are going to discuss two regional estimators of high quantiles of local distributions $F$ that are able to deal with these important features, namely, a parametric approach based on so-called two-component extreme value distributions and a semi-parametri…
In this article, we consider a jump diffusion process (X_t)observed at discrete times t=0,Delta,...,nDelta. The sampling interval Delta tends to 0 and nDelta tends to infinity. We assume that (X_t) is ergodic, strictly stationary and exponentially β-mixing. We use a penalized least-square approach to compute two adaptive estimators of the drift function b. We provide bounds for the risks of the two estimators.
We consider parametric estimation for a parabolic linear second order stochastic partial differential equation (SPDE) from high frequency data which are observed in time and space. By using thinned data obtained from the high frequency data, adaptive estimators of the coefficient parameters including the volatility parameter of a parabolic linear SPDE model are proposed. Moreover, we give some examples and simulation results of the adaptive estimators of the SPDE model based on the high frequency data.
We propose new summary statistics to quantify the association between the components in coverage-reweighted moment stationary multivariate random sets and measures. They are defined in terms of the coverage-reweighted cumulant densities and extend classic functional statistics for stationary random closed sets. We study the relations between these statistics and evaluate them explicitly for a range of models. Unbiased estimators are given for all statistics and applied to simulated examples.
We provide general adaptive upper bounds for estimating nonparametric functionals based on second order U-statistics arising from finite dimensional approximation of the infinite dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate.
This article proposes copula-based dependence quantification between multiple groups of random variables of possibly different sizes via the family of $Phi$-divergences. An axiomatic framework for this purpose is provided, after which we focus on the absolutely continuous setting assuming copula densities exist. We consider parametric and semi-parametric frameworks, discuss estimation procedures, and report on asymptotic properties of the proposed estimators. In particular, we first concentrate on a Gaussian copula approach yielding explicit and attractive dependence coefficients for specific …
This paper considers the problem of estimating a periodic function in a continuous time regression model with a general square integrable semimartingale noise. A model selection adaptive procedure is proposed. Sharp non-asymptotic oracle inequalities have been derived.
Gaussian processes that can be decomposed into a smooth mean function and a stationary autocorrelated noise process are considered and a fully automatic nonparametric method to simultaneous estimation of mean and auto-covariance functions of such processes is developed. Our empirical Bayes approach is data-driven, numerically efficient and allows for the construction of confidence sets for the mean function. Performance is demonstrated in simulations and real data analysis. The method is implemented in the R package eBsc that accompanies the paper.
It is shown that a simple Dirichlet process mixture of multivariate normals offers Bayesian density estimation with adaptive posterior convergence rates. Toward this, a novel sieve for non-parametric mixture densities is explored, and its rate adaptability to various smoothness classes of densities in arbitrary dimension is demonstrated. This sieve construction is expected to offer a substantial technical advancement in studying Bayesian non-parametric mixture models based on stick-breaking priors.
This paper is devoted to the offline multiple changes detection for long-range dependence processes. The observations are supposed to satisfy a semi-parametric long-range dependence assumption with distinct memory parameters on each stage. A penalized local Whittle contrast is considered for estimating all the parameters, notably the number of changes. The consistency as well as convergence rates are obtained. Monte-Carlo experiments exhibit the accuracy of the estimators. They also show that the estimation of the number of breaks is improved by using a data-driven slope heuristic procedure of…
Variational autoencoder with weighted samples for high-dimensional non-parametric adaptive importance sampling
ONERA（法国航空航天研究院）；Université de Toulouse（图卢兹大学）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；CNRS（法国国家科学研究中心）；Zurich University of Applied Sciences（苏黎世应用科技大学）
We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estim…
Among bivariate tail dependence measures, the tail dependence coefficient has emerged as the popular choice. Akin to the correlation matrix, a multivariate dependence measure is constructed using these bivariate measures, and this is referred to in the literature as the tail dependence matrix (TDM). While the problem of determining whether a given ${d\times d}$ matrix is a correlation matrix is of the order $O(d^3)$ in complexity, determining if a matrix is a TDM (the realization problem) is believed to be non-polynomial in complexity. Using a linear programming (LP) formulation, we show that …
An important problem in time series analysis is the discrimination between non-stationarity and longrange dependence. Most of the literature considers the problem of testing specific parametric hypotheses of non-stationarity (such as a change in the mean) against long-range dependent stationary alternatives. In this paper we suggest a simple approach, which can be used to test the null-hypothesis of a general non-stationary short-memory against the alternative of a non-stationary long-memory process. The test procedure works in the spectral domain and uses a sequence of approximating tvFARIMA …
We consider non parametric estimation problem for stochastic tomography regression model, i.e. we consider the estimation problem of function of multivariate variables (image) observed through its Radon transformation calculated with the random errors. For this problem we develop a new adaptive model selection method. By making use the Galtchouk and Pergamenshchikov approach we construct the model selection procedure for which we show a sharp non asymptotic oracle inequality for the both usual and robust quadratic risks, i.e. we show that the proposed procedure is optimal in the oracle inequal…
We propose a class of flexible non-parametric tests for the presence of dependence between components of a random vector based on weighted Cramér-von Mises functionals of the empirical copula process. The weights act as a tuning parameter and are shown to significantly influence the power of the test, making it more sensitive to different types of dependence. Asymptotic properties of the test are stated in the general case, for an arbitrary bounded and integrable weighting function, and computational formulas for a number of weighted statistics are provided. Several issues relating to the choi…
We proposed a semi-parametric estimation procedure in order to estimate the parameters of a max-mixture model and also of a max-stable model (inverse max-stable model) as an alternative to composite likelihood. A good estimation by the proposed estimator required the dependence measure to detect all dependence structures in the model, especially when dealing with the max-mixture model. We overcame this challenge by using the F-madogram. The semi-parametric estimation was then based on a quasi least square method, by minimizing the square difference between the theoretical F-madogram and an emp…
