Completely nonparametric transformation models with heteroscedastic errors are considered. Despite their flexibility, such models have rarely been used so far, since estimators of the model components have been missing and even identification of such models has not been clear until very recently. The results of Kloodt (2020) are used to construct the first two estimators of the transformation function in these models. While the first estimator converges to the true transformation function at a parametric rate, the second estimator can be obtained by an explicit formula and is less computationally demanding. Finally, a simulation study is followed by some concluding remarks. Assumptions and proofs can be found in the appendix.
The so far most general identification result in the context of nonparametric transformation models is proven. The result is constructive in the sense that it provides an explicit expression of the transformation function.
In this paper we consider a regression model that allows for time series covariates as well as heteroscedasticity with a regression function that is modelled nonparametrically. We assume that the regression function changes at some unknown time $\lfloor ns_0\rfloor$, $s_0\in(0,1)$, and our aim is to estimate the (rescaled) change point $s_0$. The considered estimator is based on a Kolmogorov-Smirnov functional of the marked empirical process of residuals. We show consistency of the estimator and prove a rate of convergence of $O_P(n^{-1})$ which in this case is clearly optimal as there are onl…
In this paper an autoregressive time series model with conditional heteroscedasticity is considered, where both conditional mean and conditional variance function are modeled nonparametrically. A test for the model assumption of independence of innovations from past time series values is suggested. The test is based on an weighted $L^2$-distance of empirical characteristic functions. The asymptotic distribution under the null hypothesis of independence is derived and consistency against fixed alternatives is shown. A smooth autoregressive residual bootstrap procedure is suggested and its perfo…
Generalized likelihood ratio statistics have been proposed in Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153-193] as a generally applicable method for testing nonparametric hypotheses about nonparametric functions. The likelihood ratio statistics are constructed based on the assumption that the distributions of stochastic errors are in a certain parametric family. We extend their work to the case where the error distribution is completely unspecified via newly proposed sieve empirical likelihood ratio (SELR) tests. The approach is also applied to test conditional estimating equations on the…
We consider the efficient estimation of the semiparametric additive transformation model with current status data. A wide range of survival models and econometric models can be incorporated into this general transformation framework. We apply the B-spline approach to simultaneously estimate the linear regression vector, the nondecreasing transformation function, and a set of nonparametric regression functions. We show that the parametric estimate is semiparametric efficient in the presence of multiple nonparametric nuisance functions. An explicit consistent B-spline estimate of the asymptotic …
The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model $Y=f(X)+σ(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ is standard normal and independent of th…
We consider partly linear transformation models applied to current status data. The unknown quantities are the transformation function, a linear regression parameter and a nonparametric regression effect. It is shown that the penalized MLE for the regression parameter is asymptotically normal and efficient and converges at the parametric rate, although the penalized MLE for the transformation function and nonparametric regression effect are only $n^{1/3}$ consistent. Inference for the regression parameter based on a block jackknife is investigated. We also study computational issues and demons…
Copula modelling has become ubiquitous in modern statistics. Here, the problem of nonparametrically estimating a copula density is addressed. Arguably the most popular nonparametric density estimator, the kernel estimator is not suitable for the unit-square-supported copula densities, mainly because it is heavily affected by boundary bias issues. In addition, most common copulas admit unbounded densities, and kernel methods are not consistent in that case. In this paper, a kernel-type copula density estimator is proposed. It is based on the idea of transforming the uniform marginals of the cop…
We propose a self-tuning $\sqrt{\mathrm {Lasso}}$ method that simultaneously resolves three important practical problems in high-dimensional regression analysis, namely it handles the unknown scale, heteroscedasticity and (drastic) non-Gaussianity of the noise. In addition, our analysis allows for badly behaved designs, for example, perfectly collinear regressors, and generates sharp bounds even in extreme cases, such as the infinite variance case and the noiseless case, in contrast to Lasso. We establish various nonasymptotic bounds for $\sqrt{\mathrm {Lasso}}$ including prediction norm rate …
This paper proposes consistent estimators for transformation parameters in semiparametric models. The problem is to find the optimal transformation into the space of models with a predetermined regression structure like additive or multiplicative separability. We give results for the estimation of the transformation when the rest of the model is estimated non- or semi-parametrically and fulfills some consistency conditions. We propose two methods for the estimation of the transformation parameter: maximizing a profile likelihood function or minimizing the mean squared distance from independenc…
We study concentration in spectral norm of nonparametric estimates of correlation matrices. We work within the confine of a Gaussian copula model. Two nonparametric estimators of the correlation matrix, the sine transformations of the Kendall's tau and Spearman's rho correlation coefficient, are studied. Expected spectrum error bound is obtained for both the estimators. A general large deviation bound for the maximum spectral error of a collection of submatrices of a given dimension is also established. These results prove that when both the number of variables and sample size are large, the s…
This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion.
The problem of nonparametric inference on a monotone function has been extensively studied in many particular cases. Estimators considered have often been of so-called Grenander type, being representable as the left derivative of the greatest convex minorant or least concave majorant of an estimator of a primitive function. In this paper, we provide general conditions for consistency and pointwise convergence in distribution of a class of generalized Grenander-type estimators of a monotone function. This broad class allows the minorization or majoratization operation to be performed on a data-…
Nous considérons dans cet article des modèles à choix binaires et coefficients aléatoires. Le but est d'estimer de manière nonparamétrique la densité du coefficient aléatoire. Il s'agit d'un problème inverse mal posé caractérisé par une transformation intégrale. Un nouvel estimateur de la densité du coefficient aléatoire est proposé. Il est basé sur les développements en séries de Fourier-Laplace sur la sphère. Cette approche permet une étude fine du problème d'identification mais aussi d'obtenir un estimateur par injection ayant une expression explicite et ne nécessitant aucun optimisation nu…
We investigate function estimation in nonparametric regression models with random design and heteroscedastic correlated noise. Adaptive properties of warped wavelet nonlinear approximations are studied over a wide range of Besov scales, $f\in\mathcal{B}^s_{π,r}$, and for a variety of $L^p$ error measures. We consider error distributions with Long-Range-Dependence parameter $α,0<α\leq1$; heteroscedasticity is modeled with a design dependent function $σ$. We prescribe a tuning paradigm, under which warped wavelet estimation achieves partial or full adaptivity results with the rates that are show…
Most results in nonparametric regression theory are developed only for the case of additive noise. In such a setting many smoothing techniques including wavelet thresholding methods have been developed and shown to be highly adaptive. In this paper we consider nonparametric regression in exponential families with the main focus on the natural exponential families with a quadratic variance function, which include, for example, Poisson regression, binomial regression and gamma regression. We propose a unified approach of using a mean-matching variance stabilizing transformation to turn the relat…
The sequential analysis of series often requires nonparametric procedures, where the most powerful ones frequently use rank transformations. Re-ranking the data sequence after each new observation can become too intensive computationally. This led to the idea of sequential ranks, where only the most recent observation is ranked. However, difficulties finding, or approximating, the null distribution of the statistics may have contributed to the lack of popularity of these methods. In this paper, we propose transforming the sequential ranks into sequential normal scores which are independent, an…
Transformation models provide a common tool for regression analysis of censored failure time data. The most common approach towards parameter estimation in these models is based on the nonparametric profile likelihood method. Several authors proposed also ad hoc M-estimators of the Euclidean component of the model. These estimators are usually simpler to impelement and many of them have good practical performance. In this paper we consider the form of the information bound for estimation if the Euclidean parameter of the model and propose a modification of inefficient M-estimators to one-step …
Consider the semiparametric transformation model $Λ_{θ_o}(Y)=m(X)+ε$, where $θ_o$ is an unknown finite dimensional parameter, the functions $Λ_{θ_o}$ and $m$ are smooth, $ε$ is independent of $X$, and $\esp(ε)=0$. We propose a kernel-type estimator of the density of the error $ε$, and prove its asymptotic normality. The estimated errors, which lie at the basis of this estimator, are obtained from a profile likelihood estimator of $θ_o$ and a nonparametric kernel estimator of $m$. The practical performance of the proposed density estimator is evaluated in a simulation study.
We study estimation of multivariate densities $p$ of the form $p(x)=h(g(x))$ for $x\in \mathbb {R}^d$ and for a fixed monotone function $h$ and an unknown convex function $g$. The canonical example is $h(y)=e^{-y}$ for $y\in \mathbb {R}$; in this case, the resulting class of densities [\mathcal {P}(e^{-y})={p=\exp(-g):g is convex}] is well known as the class of log-concave densities. Other functions $h$ allow for classes of densities with heavier tails than the log-concave class. We first investigate when the maximum likelihood estimator $\hat{p}$ exists for the class $\mathcal {P}(h)$ for var…
We present the Gaussian process density sampler (GPDS), an exchangeable generative model for use in nonparametric Bayesian density estimation. Samples drawn from the GPDS are consistent with exact, independent samples from a distribution defined by a density that is a transformation of a function drawn from a Gaussian process prior. Our formulation allows us to infer an unknown density from data using Markov chain Monte Carlo, which gives samples from the posterior distribution over density functions and from the predictive distribution on data space. We describe two such MCMC methods. Both me…
In the context of nonparametric regression models with one-sided errors, we consider parametric transformations of the response variable in order to obtain independence between the errors and the covariates. We focus in this paper on stritcly increasing and continuous transformations. In view of estimating the tranformation parameter, we use a minimum distance approach and show the uniform consistency of the estimator under mild conditions. The boundary curve, i.e. the regression function, is estimated applying a smoothed version of a local constant approximation for which we also prove the un…
Approximate Bayesian Computation is a family of likelihood-free inference techniques that are well-suited to models defined in terms of a stochastic generating mechanism. In a nutshell, Approximate Bayesian Computation proceeds by computing summary statistics s_obs from the data and simulating summary statistics for different values of the parameter theta. The posterior distribution is then approximated by an estimator of the conditional density g(theta|s_obs). In this paper, we derive the asymptotic bias and variance of the standard estimators of the posterior distribution which are based on …
Stochastic volatility modelling of financial processes has become increasingly popular. The proposed models usually contain a stationary volatility process. We will motivate and review several nonparametric methods for estimation of the density of the volatility process. Both models based on discretely sampled continuous time processes and discrete time models will be discussed. The key insight for the analysis is a transformation of the volatility density estimation problem to a deconvolution model for which standard methods exist. Three type of nonparametric density estimators are reviewed: …
Classical two-sample permutation tests for equality of distributions have exact size in finite samples, but they fail to control size for testing equality of parameters that summarize each distribution. This paper proposes permutation tests for equality of parameters that are estimated at root-$n$ or slower rates. Our general framework applies to both parametric and nonparametric models, with two samples or one sample split into two subsamples. Our tests have correct size asymptotically while preserving exact size in finite samples when distributions are equal. They have no loss in local asymp…
We consider semiparametric transformation models, where after pre-estimation of a parametric transformation of the response the data are modeled by means of nonparametric regression. We suggest subsequent procedures for testing lack-of-fit of the regression function and for significance of covariables, which - in contrast to procedures from the literature - are asymptotically not influenced by the pre-estimation of the transformation. The test statistics are asymptotically pivotal and have the same asymptotic distribution as in regression models without transformation. We show validity of a mu…
In this paper we consider a heteroscedastic transformation model, where the transformation belongs to a parametric family of monotone transformations, the regression and variance function are modelled nonparametrically and the error is independent of the multidimensional covariates. In this model, we first consider the estimation of the unknown components of the model, namely the transformation parameter, regression and variance function and the distribution of the error. We show the asymptotic normality of the proposed estimators. Second, we propose tests for the validity of the model, and es…
We tackle the problem of high-dimensional nonparametric density estimation by taking the class of log-concave densities on $\mathbb{R}^p$ and incorporating within it symmetry assumptions, which facilitate scalable estimation algorithms and can mitigate the curse of dimensionality. Our main symmetry assumption is that the super-level sets of the density are $K$-homothetic (i.e. scalar multiples of a convex body $K \subseteq \mathbb{R}^p$). When $K$ is known, we prove that the $K$-homothetic log-concave maximum likelihood estimator based on $n$ independent observations from such a density has a …
We present a new non-parametric estimator of the conditional density of the kernel type. It is based on an efficient transformation of the data by quantile transform. By use of the copula representation, it turns out to have a remarkable product form. We study its asymptotic properties and compare its bias and variance to competitors based on nonparametric regression.
