We consider the problem of estimating the conditional distribution of a post-model-selection estimator where the conditioning is on the selected model. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion such as AIC or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least-squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate this distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for this distribution. Similar impossibility results are also obtained for the conditional distribution of linear functions (e.g., predictors) of the post-model-selection estimator.
We propose an estimator of the kernel-based conditional mean dependence measure obtained from an appropriate modification of a naive estimator based on usual empirical estimators. We then get asymptotic normality of this estimator both under conditional mean independence hypothesis and under the alternative hypothesis. A new test for conditional mean independence of random variables valued into Hilbert spaces is then introduced.
We address the estimation of "extreme" conditional quantiles i.e. when their order converges to one as the sample size increases. Conditions on the rate of convergence of their order to one are provided to obtain asymptotically Gaussian distributed kernel estimators. A Weissman-type estimator and kernel estimators of the conditional tail-index are derived, permitting to estimate extreme conditional quantiles of arbitrary order.
We address the estimation of conditional quantiles when the covariate is functional and when the order of the quantiles converges to one as the sample size increases. In a first time, we investigate to what extent these large conditional quantiles can still be estimated through a functional kernel estimator of the conditional survival function. Sufficient conditions on the rate of convergence of their order to one are provided to obtain asymptotically Gaussian distributed estimators. In a second time, basing on these result, a functional Weissman estimator is derived, permitting to estimate la…
In this paper, we investigate the extreme-value methodology, to propose an improved estimator of the conditional tail expectation ($CTE$) for a loss distribution with a finite mean but infinite variance. The present work introduces a new estimator of the $CTE$ based on the bias-reduced estimators of high quantile for heavy-tailed distributions. The asymptotic normality of the proposed estimator is established and checked, in a simulation study. Moreover, we compare, in terms of bias and mean squared error, our estimator with the known old estimator.
In this paper, we consider estimation of the conditional mode of an outcome variable given regressors. To this end, we propose and analyze a computationally scalable estimator derived from a linear quantile regression model and develop asymptotic distributional theory for the estimator. Specifically, we find that the pointwise limiting distribution is a scale transformation of Chernoff's distribution despite the presence of regressors. In addition, we consider analytical and subsampling-based confidence intervals for the proposed estimator. We also conduct Monte Carlo simulations to assess the…
In cluster-specific studies, ordinary logistic regression and conditional logistic regression for binary outcomes provide maximum likelihood estimator (MLE) and conditional maximum likelihood estimator (CMLE), respectively. In this paper, we show that CMLE is approaching to MLE asymptotically when each individual data point is replicated infinitely many times. Our theoretical derivation is based on the observation that a term appearing in the conditional average log-likelihood function is the coefficient of a polynomial, and hence can be transformed to a complex integral by Cauchy's differenti…
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
In this paper, we study the local constant and the local linear estimators of the conditional density function with right-censored data which exhibit some type of dependence. It is assumed that the observations form a stationary $α-$mixing sequence. The asymptotic normality of the two estimators is established, which combined with the condition that $\lim\limits_{n\rightarrow\infty}nh_nb_n=\infty$ implies the consistency of the two estimators and can be employed to construct confidence intervals for the conditional density function. The result on the local linear estimator of the conditional d…
This paper, investigates the conditional quantile estimation of a scalar random response and a functional random covariate (i.e. valued in some infinite-dimensional space) whenever {\it functional stationary ergodic data with random censorship} are considered. We introduce a kernel type estimator of the conditional quantile function. We establish the strong consistency with rate of this estimator as well as the asymptotic normality which induces a confidence interval that is usable in practice since it does not depend on any unknown quantity. An application to electricity peak demand interval …
We find that, in a linear model, the James-Stein estimator, which dominates the maximum-likelihood estimator in terms of its in-sample prediction error, can perform poorly compared to the maximum-likelihood estimator in out-of-sample prediction. We give a detailed analysis of this phenomenon and discuss its implications. When evaluating the predictive performance of estimators, we treat the regressor matrix in the training data as fixed, i.e., we condition on the design variables. Our findings contrast those obtained by Baranchik (1973, Ann. Stat. 1:312-321) and, more recently, by Dicker (2012…
The behavior of the conditional logistic estimator is analyzed under a causal model for two-arm experimental studies with possible non-compliance in which the effect of the treatment is measured by a binary response variable. We show that, when non-compliance may only be observed in the treatment arm, the effect (measured on the logit scale) of the treatment on compliers and that of the control on non-compliers can be identified and consistently estimated under mild conditions. The same does not happen for the effect of the control on compliers. A simple correction of the conditional logistic …
Let $(X,Y)$ be a bivariate random vector. The estimation of a probability of the form $P(Y\leq y \mid X >t) $ is challenging when $t$ is large, and a fruitful approach consists in studying, if it exists, the limiting conditional distribution of the random vector $(X,Y)$, suitably normalized, given that $X$ is large. There already exists a wide literature on bivariate models for which this limiting distribution exists. In this paper, a statistical analysis of this problem is done. Estimators of the limiting distribution (which is assumed to exist) and the normalizing functions are provided, as …
When analyzing time-to-event data, it often happens that some subjects do not experience the event of interest. Survival models that take this feature into account (called `cure models') have been developed in the presence of covariates. However, the current literature on nonparametric cure models with covariates cannot be applied when the follow-up is insufficient, i.e., when the right endpoint of the support of the censoring time is strictly smaller than that of the survival time of the susceptible subjects. In this paper we attempt to fill this gap in the literature by proposing new estimat…
For discrete-time survival data, conditional likelihood inference in Cox's hazard odds model is theoretically desirable but exact calculation is numerical intractable with a moderate to large number of tied events. Unconditional maximum likelihood estimation over both regression coefficients and baseline hazard probabilities can be problematic with a large number of time intervals. We develop new methods and theory using numerically simple estimating functions, along with model-based and model-robust variance estimation, in hazard probability and odds models. For the probability hazard model, …
We study the weak convergence of conditional empirical copula processes, when the conditioning event has a nonzero probability. The validity of several bootstrap schemes is stated, including the exchangeable bootstrap. We define general - possibly conditional - multivariate dependence measures and their estimators. By applying our theoretical results, we prove the asymptotic normality of some estimators of such dependence measures.
The paper considers probability distribution, density, conditional distribution and density and conditional moments as well as their kernel estimators in spaces of generalized functions. This approach does not require restrictions on classes of distributions common in nonparametric estimation. Density in usual function spaces is not well-posed; this paper establishes existence and well-posedness of the generalized density function. It also demonstrates root-n convergence of the kernel density estimator in the space of generalized functions. It is shown that the usual kernel estimator of the co…
We consider the problem of estimating the unconditional distribution of a post-model-selection estimator. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion like AIC or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least-squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that n…
There are many interesting and widely used estimators of a functional with finite semiparametric variance bound that depend on nonparametric estimators of nuisance functions. We use cross-fitting (i.e. sample splitting) to construct novel estimators with fast remainder rates. We give cross-fit doubly robust estimators that use separate subsamples to estimate different nuisance functions. We obtain general, precise results for regression spline estimation of average linear functionals of conditional expectations with a finite semiparametric variance bound. We show that a cross-fit doubly robust…
We prove the strong consistency and the asymptotic normality of the maximum likelihood estimator of the parameters of a general conditionally heteroscedastic model with $α$-stable innovations. Then, we relax the assumptions and only suppose that the innovation process converges in distribution toward a stable process. Using a pseudo maximum likelihood estimator with a stable density, we also obtain the strong consistency and the asymptotic normality of the estimator. This framework seems relevant for financial data exhibiting heavy tails. We apply this method to several financial index and com…
Conditional density estimation (CDE) models can be useful for many statistical applications, especially because the full conditional density is estimated instead of traditional regression point estimates, revealing more information about the uncertainty of the random variable of interest. In this paper, we propose a new methodology called Odds Conditional Density Estimator (OCDE) to estimate conditional densities in a supervised learning scheme. The main idea is that it is very difficult to estimate $p_{x,y}$ and $p_{x}$ in order to estimate the conditional density $p_{y|x}$, but by introducin…
We consider inference procedures, conditional on an observed ancillary statistic, for regression coefficients under a linear regression setup where the unknown error distribution is specified nonparametrically. We establish conditional asymptotic normality of the regression coefficient estimators under regularity conditions, and formally justify the approach of plugging in kernel-type density estimators in conditional inference procedures. Simulation results show that the approach yields accurate conditional coverage probabilities when used for constructing confidence intervals. The plug-in ap…
Under a single-index regression assumption, we introduce a new semiparametric procedure to estimate a conditional density of a censored response. The regression model can be seen as a generalization of Cox regression model and also as a profitable tool to perform dimension reduction under censoring. This technique extends the results of Delecroix et al. (2003). We derive consistency and asymptotic normality of our estimator of the index parameter by proving its asymptotic equivalence with the (uncomputable) maximum likelihood estimator, using martingales results for counting processes and argu…
We propose nonparametric estimators for conditional value-at-risk (CVaR) and conditional expected shortfall (CES) associated with conditional distributions of a series of returns on a financial asset. The return series and the conditioning covariates, which may include lagged returns and other exogenous variables, are assumed to be strong mixing and follow a nonparametric conditional location-scale model. First stage nonparametric estimators for location and scale are combined with a generalized Pareto approximation for distribution tails proposed by Pickands (1975) to give final estimators fo…
This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in $x$ for all values of $z$. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be Hölder smooth in $x$, but also be total variation smooth in $z$. We propose a corresponding kernel-based estimator and prove that it achieves the mi…
We show how the problem of estimating conditional Kendall's tau can be rewritten as a classification task. Conditional Kendall's tau is a conditional dependence parameter that is a characteristic of a given pair of random variables. The goal is to predict whether the pair is concordant (value of $1$) or discordant (value of $-1$) conditionally on some covariates. We prove the consistency and the asymptotic normality of a family of penalized approximate maximum likelihood estimators, including the equivalent of the logit and probit regressions in our framework. Then, we detail specific algorith…
This paper considers extensions of minimum-disparity estimators to the problem of estimating parameters in a regression model that is conditionally specified; that is where a parametric model describes the distribution of a response $y$ conditional on covariates $x$ but does not specify the distribution of $x$. We define these estimators by estimating a non-parametric conditional density estimates and minimizing a disparity between this estimate and the parametric model averaged over values of $x$. The consistency and asymptotic normality of such estimators is demonstrated for a broad class of…
Suppose we observe a geometrically ergodic semi-Markov process and have a parametric model for the transition distribution of the embedded Markov chain, for the conditional distribution of the inter-arrival times, or for both. The first two models for the process are semiparametric, and the parameters can be estimated by conditional maximum likelihood estimators. The third model for the process is parametric, and the parameter can be estimated by an unconditional maximum likelihood estimator. We determine heuristically the asymptotic distributions of these estimators and show that they are asy…
A Bayesian estimator aiming at improving the conditional MLE is proposed by introducing a pair of priors. After explaining the conditional MLE by the posterior mode under a prior, we define a promising estimator by the posterior mean under a corresponding prior. The prior is equivalent to the reference prior in familiar models. Advantages of the present approach include two different optimality properties of the induced estimator, the ease of various extensions and the possible treatments for a finite sample size. The existing approaches are discussed and critiqued.
This paper presents a nonparametric method for estimating the conditional density associated to the jump rate of a piecewise-deterministic Markov process. In our framework, the estimation needs only one observation of the process within a long time interval. Our method relies on a generalization of Aalen's multiplicative intensity model. We prove the uniform consistency of our estimator, under some reasonable assumptions related to the primitive characteristics of the process. A simulation example illustrates the behavior of our estimator.
Entries of datasets are often collected only if an event occurred: taking a survey, enrolling in an experiment and so forth. However, such partial samples bias classical correlation estimators. Here we show how to correct for such sampling effects through two complementary estimators of event conditional correlation: the correlation of two random variables conditional on a given event. First, we provide under minimal assumptions proof of consistency and asymptotic normality for the proposed estimators. Then, through synthetic examples, we show that these estimators behave well in small-sample …
In 1991 Stute introduced a class of estimators called conditional U-statistics. They can be seen as a generalization of the Nadaraya-Watson estimator, and their strong pointwise consistency to the general regression function has been obtained in the same paper by Stute. Very recently, Gine and Mason introduced the notion of a local U-process, which generalizes that of a local empirical process, and obtained central limit theorems and laws of the iterated logarithm for this class. We apply the methods developed by Einmahl and Mason (2005) and Gine and Mason (2007a,b) to establish uniform in ban…
We consider Stochastic Volatility processes with heavy tails and possible long memory in volatility. We study the limiting conditional distribution of future events given that some present or past event was extreme (i.e. above a level which tends to infinity). Even though extremes of stochastic volatility processes are asymptotically independent (in the sense of extreme value theory), these limiting conditional distributions differ from the i.i.d. case. We introduce estimators of these limiting conditional distributions and study their asymptotic properties. If volatility has long memory, then…
Consider a high-dimensional linear regression problem, where the number of covariates is larger than the number of observations and the interest is in estimating the conditional variance of the response variable given the covariates. A conditional and unconditioned framework are considered, where conditioning is with respect to the covariates, which are ancillary to the parameter of interest. In recent papers, a consistent estimator was developed in the unconditional framework when the marginal distribution of the covariates is normal with known mean and variance. In the present work, a certai…
Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance
University of Connecticut（康涅狄格大学）；Iowa State University（爱荷华州立大学）
Subsampling is a computationally effective approach to extract information from massive data sets when computing resources are limited. After a subsample is taken from the full data, most available methods use an inverse probability weighted (IPW) objective function to estimate the model parameters. The IPW estimator does not fully utilize the information in the selected subsample. In this paper, we propose to use the maximum sampled conditional likelihood estimator (MSCLE) based on the sampled data. We established the asymptotic normality of the MSCLE and prove that its asymptotic variance co…
We consider estimating the density of a response conditioning on an error-prone covariate. Motivated by two existing kernel density estimators in the absence of covariate measurement error, we propose a method to correct the existing estimators for measurement error. Asymptotic properties of the resultant estimators under different types of measurement error distributions are derived. Moreover, we adjust bandwidths readily available from existing bandwidth selection methods developed for error-free data to obtain bandwidths for the new estimators. Extensive simulation studies are carried out t…
The problem of Voodoo correlations is recognized in neuroimaging as the problem of estimating quantities of interest from the same data that was used to select them as interesting. In statistical terminology, the problem of inference following selection from the same data is that of selective inference. Motivated by the unwelcome side-effects of the recommended remedy- splitting the data. A method for constructing confidence intervals based on the correct post-selection distribution of the observations has been suggested recently. We utilize a similar approach in order to provide point estimat…
Estimating conditional quantiles of financial time series is essential for risk management and many other applications in finance. It is well-known that financial time series display conditional heteroscedasticity. Among the large number of conditional heteroscedastic models, the generalized autoregressive conditional heteroscedastic (GARCH) process is the most popular and influential one. So far, feasible quantile regression methods for this task have been confined to a variant of the GARCH model, the linear GARCH model, owing to its tractable conditional quantile structure. This paper consid…
In extreme value analysis, the extreme value index plays a vital role as it determines the tail heaviness of the underlying distribution and is the primary parameter required for the estimation of other extreme events. In this paper, we review the estimation of the extreme value index when observations are subject to right random censoring and the presence of covariate information. In addition, we propose some estimators of the extreme value index, including a maximum likelihood estimator from a perturbed Pareto distribution. The existing estimators and the proposed ones are compared through a…
In this paper, we use quantization to construct a nonparametric estimator of conditional quantiles of a scalar response $Y$ given a d-dimensional vector of covariates $X$. First we focus on the population level and show how optimal quantization of $X$, which consists in discretizing $X$ by projecting it on an appropriate grid of $N$ points, allows to approximate conditional quantiles of $Y$ given $X$. We show that this is approximation is arbitrarily good as $N$ goes to infinity and provide a rate of convergence for the approximation error. Then we turn to the sample case and define an estimat…
