Fusion learning refers to synthesizing inferences from multiple sources or studies to provide more effective inference and prediction than from any individual source or study alone. Most existing methods for synthesizing inferences rely on parametric model assumptions, such as normality, which often do not hold in practice. In this paper, we propose a general nonparametric fusion learning framework for synthesizing inferences of the target parameter from multiple sources. The main tool underlying the proposed framework is the notion of depth confidence distribution (depth-CD), which is also developed in this paper. Broadly speaking, a depth-CD is a data-driven nonparametric summary distribution of inferential information for the target parameter. We show that a depth-CD is a useful inferential tool and, moreover, is an omnibus form of confidence regions (or p-values), whose contours of level sets shrink toward the true parameter value. The proposed fusion learning approach combines depth-CDs from the individual studies, with each depth-CD constructed by nonparametric bootstrap and data depth. This approach is shown to be efficient, general and robust. Specifically, it achieves high-order accuracy and Bahadur efficiency under suitably chosen combining elements. It allows the model or inference structure to be different among individual studies. And it readily adapts to heterogeneous studies with a broad range of complex and irregular settings. This property enables it to utilize indirect evidence from incomplete studies to gain efficiency in the overall inference. The advantages of the proposed approach are demonstrated simulations and in a Federal Aviation Administration (FAA) study of aircraft landing performance.
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
In this paper, we construct the simultaneous confidence band (SCB) for the nonparametric component in partially linear panel data models with fixed effects. We remove the fixed effects, and further obtain the estimators of parametric and nonparametric components, which do not depend on the fixed effects. We establish the asymptotic distribution of their maximum absolute deviation between the estimated nonparametric component and the true nonparametric component under some suitable conditions, and hence the result can be used to construct the simultaneous confidence band of the nonparametric co…
In this paper, we consider a weighted local linear estimator based on the inverse selection probability for nonparametric regression with missing covariates at random. The asymptotic distribution of the maximal deviation between the estimator and the true regression function is derived and an asymptotically accurate simultaneous confidence band is constructed. The estimator for the regression function is shown to be oracally efficient in the sense that it is uniformly indistinguishable from that when the selection probabilities are known. Finite sample performance is examined via simulation st…
In an empirical Bayes analysis, we use data from repeated sampling to imitate inferences made by an oracle Bayesian with extensive knowledge of the data-generating distribution. Existing results provide a comprehensive characterization of when and why empirical Bayes point estimates accurately recover oracle Bayes behavior. In this paper, we develop flexible and practical confidence intervals that provide asymptotic frequentist coverage of empirical Bayes estimands, such as the posterior mean or the local false sign rate. The coverage statements hold even when the estimands are only partially …
In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and non-asymptotic confidence region which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically centre on certain simplest functions in that region where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the co…
Many recently developed Bayesian methods have focused on sparse signal detection. However, much less work has been done addressing the natural follow-up question: how to make valid inferences for the magnitude of those signals after selection. Ordinary Bayesian credible intervals suffer from selection bias, owing to the fact that the target of inference is chosen adaptively. Existing Bayesian approaches for correcting this bias produce credible intervals with poor frequentist properties, while existing frequentist approaches require sacrificing the benefits of shrinkage typical in Bayesian met…
Since Stein's 1956 seminal paper, shrinkage has played a fundamental role in both parametric and nonparametric inference. This article discusses minimaxity and adaptive minimaxity in nonparametric function estimation. Three interrelated problems, function estimation under global integrated squared error, estimation under pointwise squared error, and nonparametric confidence intervals, are considered. Shrinkage is pivotal in the development of both the minimax theory and the adaptation theory. While the three problems are closely connected and the minimax theories bear some similarities, the ad…
Fiducial Inference, introduced by Fisher in the 1930s, has a long history, which at times aroused passionate disagreements. However, its application has been largely confined to relatively simple parametric problems. In this paper, we present what might be the first time fiducial inference, as generalized by Hannig et al. (2016), is systematically applied to estimation of a nonparametric survival function under right censoring. We find that the resulting fiducial distribution gives rise to surprisingly good statistical procedures applicable to both one sample and two sample problems. In partic…
We develop a novel method of constructing confidence bands for nonparametric regression functions under shape constraints. This method can be implemented via a linear programming, and it is thus computationally appealing. We illustrate a usage of our proposed method with an application to the regression kink design (RKD). Econometric analyses based on the RKD often suffer from wide confidence intervals due to slow convergence rates of nonparametric derivative estimators. We demonstrate that economic models and structures motivate shape restrictions, which in turn contribute to shrinking the co…
In international football (soccer), two-legged knockout ties, with each team playing at home in one leg and the final outcome decided on aggregate, are common. Many players, managers and followers seem to believe in the `second-leg home advantage', i.e. that it is beneficial to play at home on the second leg. A more complex effect than the usual and well-established home advantage, it is harder to identify, and previous statistical studies did not prove conclusive about its actuality. Yet, given the amount of money handled in international football competitions nowadays, the question of existe…
A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic coverage guarantees under nonparametric conditions. We draw connections between the Cramér-Chernoff method for exponential concentration, the law of the iterated logarithm (LIL), and the sequential probability ratio test -- our confidence sequences are time-uniform extensions of the first; provide tight, nonasymptotic characterizations of the second; and generalize the third to nonparametric settin…
We propose nonparametric estimation of divergence measures between continuous distributions. Our approach is based on a plug-in kernel- type estimators of density functions. We give the uniform in bandwidth consistency for the proposal estimators. As a consequence, their asymp- totic 100% confidence intervals are also provided.
We consider nonparametric regression in the context of functional data, that is, when a random sample of functions is observed on a fine grid. We obtain a functional asymptotic normality result allowing to build simultaneous confidence bands (SCB) for various estimation and inference tasks. Two applications to a SCB procedure for the regression function and to a goodness-of-fit test for curvilinear regression models are proposed. The first one has improved accuracy upon the other available methods while the second can detect local departures from a parametric shape, as opposed to the usual goo…
This paper develops inferential methods for a very general class of ill-posed models in econometrics encompassing the nonparametric instrumental variable regression, various functional regressions, and the density deconvolution. We focus on uniform confidence sets for the parameter of interest estimated with Tikhonov regularization, as in Darolles, Fan, Florens, and Renault (2011). Since it is impossible to have inferential methods based on the central limit theorem, we develop two alternative approaches relying on the concentration inequality and bootstrap approximations. We show that expecte…
Nonparametric statistics for distribution functions F or densities f=F' under qualitative shape constraints provides an interesting alternative to classical parametric or entirely nonparametric approaches. We contribute to this area by considering a new shape constraint: F is said to be bi-log-concave, if both log(F) and log(1 - F) are concave. Many commonly considered distributions are compatible with this constraint. For instance, any c.d.f. F with log-concave density f = F' is bi-log-concave. But in contrast to the latter constraint, bi-log-concavity allows for multimodal densities. We prov…
Statistical inference in high dimensional settings has recently attracted enormous attention within the literature. However, most published work focuses on the parametric linear regression problem. This paper considers an important extension of this problem: statistical inference for high dimensional sparse nonparametric additive models. To be more precise, this paper develops a methodology for constructing a probability density function on the set of all candidate models. This methodology can also be applied to construct confidence intervals for various quantities of interest (such as noise v…
In this paper, we propose to construct confidence bands by bootstrapping the debiased kernel density estimator (for density estimation) and the debiased local polynomial regression estimator (for regression analysis). The idea of using a debiased estimator was recently employed by Calonico et al. (2018b) to construct a confidence interval of the density function (and regression function) at a given point by explicitly estimating stochastic variations. We extend their ideas of using the debiased estimator and further propose a bootstrap approach for constructing simultaneous confidence bands. T…
We propose two classes of nonparametric point estimators of $θ=P(X<Y)$ in the case where $(X,Y)$ are paired, possibly dependent, absolutely continuous random variables. The proposed estimators are based on nonparametric estimators of the joint density of $(X,Y)$ and the distribution function of $Z=Y - X$. We explore the use of several density and distribution function estimators and characterise the convergence of the resulting estimators of $θ$. We consider the use of bootstrap methods to obtain confidence intervals. The performance of these estimators is illustrated using simulated and real …
Carnegie Mellon University（卡内基梅隆大学）；University of Rome（罗马大学）
We derive nonparametric confidence intervals for the eigenvalues of the Hessian at modes of a density estimate. This provides information about the strength and shape of modes and can also be used as a significance test. We use a data-splitting approach in which potential modes are identified using the first half of the data and inference is done with the second half of the data. To get valid confidence sets for the eigenvalues, we use a bootstrap based on an elementary-symmetric-polynomial (ESP) transformation. This leads to valid bootstrap confidence sets regardless of any multiplicities in …
We consider inference procedures, conditional on an observed ancillary statistic, for regression coefficients under a linear regression setup where the unknown error distribution is specified nonparametrically. We establish conditional asymptotic normality of the regression coefficient estimators under regularity conditions, and formally justify the approach of plugging in kernel-type density estimators in conditional inference procedures. Simulation results show that the approach yields accurate conditional coverage probabilities when used for constructing confidence intervals. The plug-in ap…
This study examines a nonparametric inference on a stationary Lévy-driven Ornstein-Uhlenbeck (OU) process $X = (X_{t})_{t \geq 0}$ with a compound Poisson subordinator. We propose a new spectral estimator for the Lévy measure of the Lévy-driven OU process $X$ under macroscopic observations. We also derive, for the estimator, multivariate central limit theorems over a finite number of design points, and high-dimensional central limit theorems in the case wherein the number of design points increases with an increase in the sample size. Built on these asymptotic results, we develop methods to co…
Inference procedures in sequential trial emulation with survival outcomes: comparing confidence intervals based on the sandwich variance estimator, bootstrap and jackknife
Nonparametric two-stage procedures to construct fixed-width confidence intervals are studied to quantify uncertainty. It is shown that the validity of the random central limit theorem (RCLT) accompanied by a consistent and asymptotically unbiased estimator of the asymptotic variance already guarantees consistency and first as well as second order efficiency of the two-stage procedures. This holds under the common asymptotics where the length of the confidence interval tends to $0$ as well as under the novel proposed high-confident asymptotics where the confidence level tends to $1$. The approa…
We develop a nonparametric extension of the sequential generalized likelihood ratio (GLR) test and corresponding time-uniform confidence sequences for the mean of a univariate distribution. By utilizing a geometric interpretation of the GLR statistic, we derive a simple analytic upper bound on the probability that it exceeds any prespecified boundary; these are intractable to approximate via simulations due to infinite horizon of the tests and the composite nonparametric nulls under consideration. Using time-uniform boundary-crossing inequalities, we carry out a unified nonasymptotic analysis …
Fiducial inference, as generalized by Hannig et al. (2016), is applied to nonparametric g-modeling (Efron, 2016) in the discrete case. We propose a computationally efficient algorithm to sample from the fiducial distribution, and use the generated samples to construct point estimates and confidence intervals. We study the theoretical properties of the fiducial distribution and perform extensive simulations in various scenarios. The proposed approach yields good statistical performance in terms of the mean squared error of point estimators and the coverage of confidence intervals. Furthermore, …
We provide confidence bands for isotonic quantile curves in nonparametric univariate regression with guaranteed given coverage probability. The method is an adaptation of the confidence bands of Duembgen and Johns (2004) for isotonic median curves.
New methods and theory have recently been developed to nonparametrically estimate cumulative incidence functions for competing risks survival data subject to current status censoring. In particular, the limiting distribution of the nonparametric maximum likelihood estimator and a simplified "naive estimator" have been established under certain smoothness conditions. In this paper, we establish the large-sample behavior of these estimators in two additional models, namely when the observation time distribution has discrete support and when the observation times are grouped. These asymptotic res…
We consider the estimation of the total number $N$ of species based on the abundances of species that have been observed. We adopt a non parametric approach where the true abundance distribution $p$ is only supposed to be convex. From this assumption, we propose a definition for convex abundance distributions. We use a least-squares estimate of the truncated version of $p$ under the convexity constraint. We deduce two estimators of the total number of species, the asymptotic distribution of which are derived. We propose three different procedures, including a bootstrap one, to obtain a confide…
In this paper we propose an autoregressive wild bootstrap method to construct confidence bands around a smooth deterministic trend. The bootstrap method is easy to implement and does not require any adjustments in the presence of missing data, which makes it particularly suitable for climatological applications. We establish the asymptotic validity of the bootstrap method for both pointwise and simultaneous confidence bands under general conditions, allowing for general patterns of missing data, serial dependence and heteroskedasticity. The finite sample properties of the method are studied in…
During the last decades there has been increasing interest in modeling the volatility of financial data. Several parametric models have been proposed to this aim, starting from ARCH, GARCH and their variants, but often it is hard to evaluate which one is the most suitable for the analyzed financial data. In this paper we focus on nonparametric analysis of the volatility function for mixing processes. Our approach encompasses many parametric frameworks and supplies several tools which can be used to give evidence against or in favor of a specific parametric model: nonparametric function estimat…
The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functional parameters by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where …
Understanding the pathways whereby an intervention has an effect on an outcome is a common scientific goal. A rich body of literature provides various decompositions of the total intervention effect into pathway specific effects. Interventional direct and indirect effects provide one such decomposition. Existing estimators of these effects are based on parametric models with confidence interval estimation facilitated via the nonparametric bootstrap. We provide theory that allows for more flexible, possibly machine learning-based, estimation techniques to be considered. In particular, we establ…
We present a new approach for inference about a log-concave distribution: Instead of using the method of maximum likelihood, we propose to incorporate the log-concavity constraint in an appropriate nonparametric confidence set for the cdf $F$. This approach has the advantage that it automatically provides a measure of statistical uncertainty and it thus overcomes a marked limitation of the maximum likelihood estimate. In particular, we show how to construct confidence bands for the density that have a finite sample guaranteed confidence level. The nonparametric confidence set for $F$ which we …
The advent and subsequent widespread availability of preventive vaccines has altered the course of public health over the past century. Despite this success, effective vaccines to prevent many high-burden diseases, including HIV, have been slow to develop. Vaccine development can be aided by the identification of immune response markers that serve as effective surrogates for clinically significant infection or disease endpoints. However, measuring immune response marker activity is often costly, which has motivated the usage of two-phase sampling for immune response evaluation in clinical tria…
The construction of confidence regions for parameter vectors is a difficult problem in the nonparametric setting, particularly when the sample size is not large. The bootstrap has shown promise in solving this problem, but empirical evidence often indicates that some bootstrap methods have difficulty in maintaining the correct coverage probability, while other methods may be unstable, often resulting in very large confidence regions. One way to improve the performance of a bootstrap confidence region is to restrict the shape of the region in such a way that the error term of an expansion is as…
Pattern-mixture models provide a transparent approach for handling missing data, where the full-data distribution is factorized in a way that explicitly shows the parts that can be estimated from observed data alone, and the parts that require identifying restrictions. We introduce a nonparametric estimator of the full-data distribution based on the pattern-mixture model factorization. Our approach uses the empirical observed-data distribution and augments it with a nonparametric estimator of the missing-data distributions under a given identifying restriction. Our results apply to a large cla…
Using and extending fractional order statistic theory, we characterize the $O(n^{-1})$ coverage probability error of the previously proposed confidence intervals for population quantiles using $L$-statistics as endpoints in Hutson (1999). We derive an analytic expression for the $n^{-1}$ term, which may be used to calibrate the nominal coverage level to get $O\bigl(n^{-3/2}[\log(n)]^3\bigr)$ coverage error. Asymptotic power is shown to be optimal. Using kernel smoothing, we propose a related method for nonparametric inference on conditional quantiles. This new method compares favorably with as…
