We consider least squares estimators of the finite regression parameter $α$ in the single index regression model $Y=ψ(α^T X)+ε$, where $X$ is a $d$-dimensional random vector, $\E(Y|X)=ψ(α^T X)$, and where $ψ$ is monotone. It has been suggested to estimate $α$ by a profile least squares estimator, minimizing $\sum_{i=1}^n(Y_i-ψ(α^T X_i))^2$ over monotone $ψ$ and $α$ on the boundary $S_{d-1}$of the unit ball. Although this suggestion has been around for a long time, it is still unknown whether the estimate is $\sqrt{n}$ convergent. We show that a profile least squares estimator, using the same pointwise least squares estimator for fixed $α$, but using a different global sum of squares, is $\sqrt{n}$-convergent and asymptotically normal. The difference between the corresponding loss functions is studied and also a comparison with other methods is given.
A monotone version of an empirical Bayes estimator for the parameter of the Borel-Tanner distribution is constructed. Some properties of the estimator's regret risk are illustrated through simulations.
We consider the smoothed maximum likelihood estimator and the smoothed Grenander-type estimator for a monotone baseline hazard rate $λ_0$ in the Cox model. We analyze their asymptotic behavior and show that they are asymptotically normal at rate $n^{m/(2m+1)}$, when~$λ_0$ is $m\geq 2$ times continuously differentiable, and that both estimators are asymptotically equivalent. Finally, we present numerical results on pointwise confidence intervals that illustrate the comparable behavior of the two methods.
We propose a new estimator of a discrete monotone probability mass function with known flat regions. We analyse its asymptotic properties and compare its performance to the Grenander estimator and to the monotone rearrangement estimator.
In this paper we will consider the estimation of a monotone regression (or density) function in a fixed point by the least squares (Grenander) estimator. We will show that this estimator is fully adaptive, in the sense that the attained rate is given by a functional relation using the underlying function $f_0$, and not by some smoothness parameter, and that this rate is optimal when considering the class of all monotone functions, in the sense that there exists a sequence of alternative monotone functions $f_1$, such that no other estimator can attain a better rate for both $f_0$ and $f_1$. We…
We consider two isotonic smooth estimators for a monotone baseline hazard in the Cox model, a maximum smooth likelihood estimator and a Grenander-type estimator based on the smoothed Breslow estimator for the cumulative baseline hazard. We show that they are both asymptotically normal at rate $n^{m/(2m+1)}$, where $m\geq 2$ denotes the level of smoothness considered, and we relate their limit behavior to kernel smoothed isotonic estimators studied in Lopuhaä and Musta (2016). It turns out that the Grenander-type estimator is asymptotically equivalent to the kernel smoothed isotonic estimators,…
In a multi-index model with $k$ index vectors, the input variables are transformed by taking inner products with the index vectors. A transfer function $f: \mathbb{R}^k \to \mathbb{R}$ is applied to these inner products to generate the output. Thus, multi-index models are a generalization of linear models. In this paper, we consider monotone multi-index models. Namely, the transfer function is assumed to be coordinate-wise monotone. The monotone multi-index model therefore generalizes both linear regression and isotonic regression, which is the estimation of a coordinate-wise monotone function…
We study the problem of estimating piecewise monotone vectors. This problem can be seen as a generalization of the isotonic regression that allows a small number of order-violating changepoints. We focus mainly on the performance of the nearly-isotonic regression proposed by Tibshirani et al. (2011). We derive risk bounds for the nearly-isotonic regression estimators that are adaptive to piecewise monotone signals. The estimator achieves a near minimax convergence rate over certain classes of piecewise monotone signals under a weak assumption. Furthermore, we present an algorithm that can be a…
In many problems, a sensible estimator of a possibly multivariate monotone function may itself fail to be monotone. We study the correction of such an estimator obtained via projection onto the space of functions monotone over a finite grid in the domain. We demonstrate that this corrected estimator has no worse supremal estimation error than the initial estimator, and that analogously corrected confidence bands contain the true function whenever the initial bands do, at no loss to average or maximal band width. Additionally, we demonstrate that the corrected estimator is uniformly asymptotica…
We discuss a number of estimates of the hazard under the assumption that the hazard is monotone on an interval [0,a]. The usual isotonic least squares estimators of the hazard are inconsistent at the boundary points 0 and a. We use penalization to obtain uniformly consistent estimators. Moreover, we determine the optimal penalization constants, extending related work in this direction by Woodroofe and Sun (1993) and Woodroofe and Sun (1999). Two methods of obtaining smooth monotone estimates based on a non-smooth monotone estimator are discussed. One is based on kernel smoothing, the other on …
We propose two estimators of a monotone spectral density, that are based on the periodogram. These are the isotonic regression of the periodogram and the isotonic regression of the log-periodogram. We derive pointwise limit distribution results for the proposed estimators for short memory linear processes and long memory Gaussian processes and also that the estimators are rate optimal.
In this paper, the model $Y_i=g(Z_i),\ i=1,2,...,n$ with $Z_i$ being random variables with known distribution and $g(x)$ being unknown strictly increasing function is proposed and almost sure convergence of estimator for $g(x)$ is proved for i.i.d and short range dependent data. Confidence intervals and bands are constructed for i.i.d data theoretically and confidence intervals are introduced for short range dependent data through resampling. Besides, a test for equivalence of $g(x)$ to the desired function is proposed. Finite sample analysis and application of this model on an urban waste wat…
We study a class of nonlinear nonparametric inverse problems. Specifically, we propose a nonparametric estimator of the dynamics of a monotonically increasing trajectory defined on a finite time interval. Under suitable regularity conditions, we prove consistency of the proposed estimator and show that in terms of $L^2$-loss, the optimal rate of convergence for the proposed estimator is the same as that for the estimation of the derivative of a trajectory. This is a new contribution to the area of nonlinear nonparametric inverse problems. We conduct a simulation study to examine the finite sam…
We consider Grenander type estimators for monotone functions $f$ in a very general setting, which includes estimation of monotone regression curves, monotone densities, and monotone failure rates. These estimators are defined as the left-hand slope of the least concave majorant $\hat{F}_n$ of a naive estimator $F_n$ of the integrated curve $F$ corresponding to $f$. We prove that the supremum distance between $\hat{F}_n$ and $F_n$ is of the order $O_p(n^{-1}\log n)^{2/(4-τ)}$, for some $τ\in[0,4)$ that characterizes the tail probabilities of an approximating process for $F_n$. In typical exampl…
We study and compare three estimators of a discrete monotone distribution: (a) the (raw) empirical estimator; (b) the "method of rearrangements" estimator; and (c) the maximum likelihood estimator. We show that the maximum likelihood estimator strictly dominates both the rearrangement and empirical estimators in cases when the distribution has intervals of constancy. For example, when the distribution is uniform on $\{0, ..., y \}$, the asymptotic risk of the method of rearrangements estimator (in squared $\ell_2$ norm) is $y/(y+1)$, while the asymptotic risk of the MLE is of order $(\log y)/(…
Shape constrained densities are encountered in many nonparametric estimation problems. The classes of monotone or convex (and monotone) densities can be viewed as special cases of the classes of k-monotone densities. A density g is said to be k-monotone if its successive derivatives up to order k-2 are alternately negative or positive, with the k-2 derivative convex (or concave) according as k is even (or odd) respectively. g is simply nonincreasing if k=1. These classes of shape constrained densities bridge the gap between the classes of monotone (1-monotone) and convex decreasing (2-monotone…
We study the monotone single index model where a real response variable $Y $ is linked to a $d$-dimensional covariate $X$ through the relationship $E[Y | X] = Ψ_0(α^T_0 X)$ almost surely. Both the ridge function, $Ψ_0$, and the index parameter, $α_0$, are unknown and the ridge function is assumed to be monotone on its interval of support. Under some regularity conditions, without imposing a particular distribution on the regression error, we show the $n^{-1/3}$ rate of convergence in the $\ell_2$-norm for the least squares estimator of the bundled function $ψ_0(α^T_0 \cdot),$ and also that of …
Random samples are lossy summaries which allow queries posed over the data to be approximated by applying an appropriate estimator to the sample. The effectiveness of sampling, however, hinges on estimator selection. The choice of estimators is subjected to global requirements, such as unbiasedness and range restrictions on the estimate value, and ideally, we seek estimators that are both efficient to derive and apply and {\em admissible} (not dominated, in terms of variance, by other estimators). Nevertheless, for a given data domain, sampling scheme, and query, there are many admissible esti…
We consider estimation in the single index model where the link function is monotone. For this model a profile least squares estimator has been proposed to estimate the unknown link function and index. Although it is natural to propose this procedure, it is still unknown whether it produces index estimates which converge at the parametric rate. We show that this holds if we solve a score equation corresponding to this least squares problem. Using a Lagrangian formulation, we show how one can solve this score equation without any reparametrization. This makes it easy to solve the score equation…
The problem of nonparametric inference on a monotone function has been extensively studied in many particular cases. Estimators considered have often been of so-called Grenander type, being representable as the left derivative of the greatest convex minorant or least concave majorant of an estimator of a primitive function. In this paper, we provide general conditions for consistency and pointwise convergence in distribution of a class of generalized Grenander-type estimators of a monotone function. This broad class allows the minorization or majoratization operation to be performed on a data-…
In the paper we propose some new class of functions which is used to construct tail index estimators. Functions from this new class is non-monotone in general, but presents a product of two monotone functions: the power function and the logarithmic function, which plays essential role in the classical Hill estimator. Introduced new estimators have better asymptotic performance comparing with the Hill estimator and other popular estimators over all range of the parameters present in the second order regular variation condition. Asymptotic normality of the introduced estimators is proved, and co…
We consider Grenander type estimators for a monotone function $λ:[0,1]\to\mathbb{R}$, obtained as the slope of a concave (convex) estimate of the primitive of $λ$. Our main result is a central limit theorem for the Hellinger loss, which applies to statistical models that satisfy the setup in Durot (2007). This includes estimation of a monotone density, for which the limiting variance of the Hellinger loss turns out to be independent of $λ$.
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic estimators obtained by smoothing a Grenander-type estimator or isotonizing the ordinary kernel estimator. As a preliminary result we establish a similar result for ordinary kernel estimators. Our results are obtained in a general setting, which includes estimation of a monotone density, regression function and hazard rate. We also perform a simulation study for te…
In the standard setting of one-way ANOVA with normal errors, a new algorithm, called the Step Down Maximum Mean Selection Algorithm (SDMMSA), is proposed to estimate the treatment means under an assumption that the treatment mean is nondecreasing in the factor level. We prove that i) the SDMMSA and the Pooled Adjacent Violator Algorithm (PAVA), a widely used algorithm in many problems, generate the same estimators for normal means, ii) the estimators are the mle's, and iii) the distribution of each of the estimators is stochastically nondecreasing in each of the treatment means. As an applicat…
A class of random hazard rates, which is defined as a mixture of an indicator kernel convolved with a completely random measure, is of interest. We provide an explicit characterization of the posterior distribution of this mixture hazard rate model via a finite mixture of S-paths. A closed and tractable Bayes estimator for the hazard rate is derived to be a finite sum over S-paths. The path characterization or the estimator is proved to be a Rao--Blackwellization of an existing partition characterization or partition-sum estimator. This accentuates the importance of S-paths in Bayesian modelin…
The monotone rearrrangement algorithm was introduced by Hardy, Littlewood and Pólya as a sorting device for functions. Assuming that $x$ is a monotone function and that an estimate $x_n$ of $x$ is given, consider the monotone rearrangement $\hat{x}_n$ of $x_n$. This new estimator is shown to be uniformly consistent. Under suitable assumptions, pointwise limit distribution results for $\hat{x}_n$ are obtained. The framework is general and allows for weakly dependent and long range dependent stationary data. Applications in monotone density and regression function estimation are detailed.
The problem of estimating the probability p=P(g(X<0) is considered when X represents a multivariate stochastic input of a monotone function g. First, a heuristic method to bound p is formally described, involving a specialized design of numerical experiments. Then a statistical estimation of p is considered based on a sequential stochastic exploration of the input space. A maximum likelihood estimator of p based on successive dependent Bernoulli data is defined and its theoretical convergence properties are studied. Under intuitive or mild conditions, the estimation is faster and more robust t…
The k-monotone classes of densities defined on (0, \infty) have been known in the mathematical literature but were for the first time considered from a statistical point of view by Balabdaoui and Wellner (2007, 2010). In these works, the authors generalized the results established for monotone (k=1) and convex (k=2) densities by giving a characterization of the Maximum Likelihood and Least Square estimators (MLE and LSE) and deriving minimax bounds for rates of convergence. For k strictly larger than 2, the pointwise asymptotic behavior of the MLE and LSE studied by Balabdaoui and Wellner (200…
In this paper, we consider the situation in which the observations follow an isotonic generalized partly linear model. Under this model, the mean of the responses is modelled, through a link function, linearly on some covariates and nonparametrically on an univariate regressor in such a way that the nonparametric component is assumed to be a monotone function. A class of robust estimates for the monotone nonparametric component and for the regression parameter, related to the linear one, is defined. The robust estimators are based on a spline approach combined with a score function which bound…
Motivated by global warming issues, we consider a time se- ries that consists of a nondecreasing trend observed with station- ary fluctuations, nonparametric estimation of the trend under monotonicity assumption is considered. The rescaled isotonic es- timators at an interior point are shown to converge to Chernoff's distribution under minimal conditions on the stationary errors. Since the isotonic estimators suffer from the spiking problem at the end point, two modifications are proposed. The estima- tion errors for both estimators of the boundary point are shown to have interesting limiting …
An estimator for the $\M$-index of functions of $\mathcal{M}$, a larger class than the class of regularly varying (RV) functions, is proposed. This index is the tail index of RV functions and this estimator is thus a new one on the class of RV functions. This estimator satisfies, assuming suitable conditions, strong consistency. Asymptotic normality of this estimator is proved for a large class of RV functions, showing a better performance than some well-known estimators. Illustrations with simulated and real life datasets are provided.
We derive minimax generalized Bayes estimators of regression coefficients in the general linear model with spherically symmetric errors under invariant quadratic loss for the case of unknown scale. The class of estimators generalizes the class considered in Maruyama and Strawderman (2005) to include non-monotone shrinkage functions.
We propose a class of weighted least squares estimators for the tail index of a distribution function with a regularly varying upper tail. Our approach is based on the method developed by \cite{Holan2010} for the Parzen tail index. Asymptotic normality of the estimators is proved. Through a simulation study, these and earlier estimators are compared in the Pareto and Hall models using the mean squared error as criterion. The results show that the weighted least squares estimator is better than the other estimators investigated.
In this paper we continue the investigation of the SRCEN estimator of the extreme value index $γ$ (or the tail index $α=1/γ$) proposed in \cite{MCE} for $γ>1/2$. We propose a new estimator based on the local maximum. This, in fact, is a modification of the SRCEN estimator to the case $γ>0$. We establish the consistency and asymptotic normality of the newly proposed estimator for i.i.d. data. Also, a short discussion on the comparison of the estimators is included.
We consider a high-dimensional monotone single index model (hdSIM), which is a semiparametric extension of a high-dimensional generalize linear model (hdGLM), where the link function is unknown, but constrained with monotone and non-decreasing shape. We develop a scalable projection-based iterative approach, the "Sparse Orthogonal Descent Single-Index Model" (SOD-SIM), which alternates between sparse-thresholded orthogonalized "gradient-like" steps and isotonic regression steps to recover the coefficient vector. Our main contribution is that we provide finite sample estimation bounds for both …
Estimation of the tail index of heavy-tailed distributions and its applications are essential in many research areas. We propose a class of weighted least squares (WLS) estimators for the Parzen tail index. Our approach is based on the method developed by \cite{Holan2010}. We investigate consistency and asymptotic normality of the WLS estimators. Through a simulation study, we make a comparison with the Hill, Pickands, DEdH (Dekkers, Einmahl and de Haan) and ordinary least squares (OLS) estimators using the mean square error as criterion. The results show that in a restricted model some member…
We aim at estimating a function $λ:[0,1]\to \mathbb {R}$, subject to the constraint that it is decreasing (or increasing). We provide a unified approach for studying the $\mathbb {L}_p$-loss of an estimator defined as the slope of a concave (or convex) approximation of an estimator of a primitive of $λ$, based on $n$ observations. Our main task is to prove that the $\mathbb {L}_p$-loss is asymptotically Gaussian with explicit (though unknown) asymptotic mean and variance. We also prove that the local $\mathbb {L}_p$-risk at a fixed point and the global $\mathbb {L}_p$-risk are of order $n^{-p/…
By means of a Lynden-Bell integral with deterministic threshold, Worms and Worms [A Lynden-Bell integral estimator for extremes of randomly truncated data. Statist. Probab. Lett. 2016; 109: 106-117] recently introduced an asymptotically normal estimator of the tail index for randomly right-truncated Pareto-type data. In this context, we consider the random threshold case to derive a Hill-type estimator and establish its consistency and asymptotic normality. A simulation study is carried out to evaluate the finite sample behavior of the proposed estimator.
The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…
In this paper we introduce a new class of diffeomorphic smoothers based on general spline smoothing techniques and on the use of some tools that have been recently developed in the context of image warping to compute smooth diffeomorphisms. This diffeomorphic spline is defined as the solution of an ordinary differential equation governed by an appropriate time-dependent vector field. This solution has a closed form expression which can be computed using classical unconstrained spline smoothing techniques. This method does not require the use of quadratic or linear programming under inequality …
Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g(<a,x>)$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that …
In this paper, we investigate the problem of nonparametric monotone frontier estimation from the perspective of extreme value theory. This enables us to revisit the asymptotic theory of the popular free disposal hull estimator in a more general setting, to derive new and asymptotically Gaussian estimators and to provide useful asymptotic confidence bands for the monotone boundary function. The finite-sample behavior of the suggested estimators is explored via Monte Carlo experiments. We also apply our approach to a real data set based on the production activity of the French postal services.
We derive oracle inequalities for the problems of isotonic and convex regression using the combination of $Q$-aggregation procedure and sparsity pattern aggregation. This improves upon the previous results including the oracle inequalities for the constrained least squares estimator. One of the improvements is that our oracle inequalities are sharp, i.e., with leading constant 1. It allows us to obtain bounds for the minimax regret thus accounting for model misspecification, which was not possible based on the previous results. Another improvement is that we obtain oracle inequalities both wit…
This paper addresses the problem of estimating the extreme value index in presence of random censoring for distributions in the Weibull domain of attraction. The methodologies introduced in [Worms (2014)], in the heavy-tailed case, are adapted here to the negative extreme value index framework, leading to the definition of weighted versions of the popular moments of relative excesses with arbitrary exponent. This leads to the definition of two families of estimators (with an adaptation of the so called Moment estimator as a particular case), for which the consistency is proved under a first or…
We study the asymptotic behavior of the Maximum Likelihood and Least Squares Estimators of a $k$-monotone density $g_0$ at a fixed point $x_0$ when $k>2$. We find that the $j$th derivative of the estimators at $x_0$ converges at the rate $n^{-(k-j)/(2k+1)}$ for $j=0,...,k-1$. The limiting distribution depends on an almost surely uniquely defined stochastic process $H_k$ that stays above (below) the $k$-fold integral of Brownian motion plus a deterministic drift when $k$ is even (odd). Both the MLE and LSE are known to be splines of degree $k-1$ with simple knots. Establishing the order of the …
In extreme value statistics for stationary sequences, blocks estimators are usually constructed by using disjoint blocks because exceedances over high thresholds of different blocks can be assumed asymptotically independent. In this paper we focus on the estimation of the extremal index which measures the degree of clustering of extremes. We consider disjoint and sliding blocks estimators and compare their asymptotic properties. In particular we show that the sliding blocks estimator is more efficient than the disjoint version and has a smaller asymptotic bias. Moreover we propose a method to …
Stochastic simulators such as Monte-Carlo estimators are widely used in science and engineering to study physical systems through their probabilistic representation. Global sensitivity analysis aims to identify the input parameters which have the most important impact on the output. A popular tool to perform global sensitivity analysis is the variance-based method which comes from the Hoeffding-Sobol decomposition. Nevertheless, this method requires an important number of simulations and is often unfeasible under reasonable time constraint. Therefore, an approximation of the input/output relat…
We make use of the empirical process theory to approximate the adapted Hill estimator, for censored data, in terms of Gaussian processes. Then, we derive its asymptotic normality, only under the usual second-order condition of regular variation. Our methodology allows to relax the assumptions, made in Einmahl, Fils-Villetard and Guillou(2008), on the heavy-tailed distribution functions and the sample fraction of upper order statistics.
The extremal index $θ$, a number in the interval $[0,1]$, is known to be a measure of primal importance for analyzing the extremes of a stationary time series. New rank-based estimators for $θ$ are proposed which rely on the construction of approximate samples from the exponential distribution with parameter $θ$ that is then to be fitted via the method of moments. The new estimators are analyzed both theoretically as well as empirically through a large-scale simulation study. In specific scenarios, in particular for time series models with $θ\approx 1$, they are found to be superior to recent …
A tail empirical process for heavy-tailed and right-censored data is introduced and its Gaussian approximation is established. In this context, a (weighted) new Hill-type estimator for positive extreme value index is proposed and its consistency and asymptotic normality are proved by means of the aforementioned process in the framework of second-order conditions of regular variation. In a comparative simulation study, the newly defined estimator is seen to perform better than the already existing ones in terms of both bias and mean squared error. As a real data example, we apply our estimation…
We propose a two-step pseudo-maximum likelihood procedure for semiparametric single-index regression models where the conditional variance is a known function of the regression and an additional parameter. The Poisson single-index regression with multiplicative unobserved heterogeneity is an example of such models. Our procedure is based on linear exponential densities with nuisance parameter. The pseudo-likelihood criterion we use contains a nonparametric estimate of the index regression and therefore a rule for choosing the smoothing parameter is needed. We propose an automatic and natural r…
We consider heavy-tailed distributions and compare the well-known estimators of the tail index, based on extreme value theory with a comparatively recent estimator based on a different idea.
