The analysis of continuously spatially varying processes usually considers two sources of variation, namely, the large-scale variation collected by the trend of the process, and the small-scale variation. Parametric trend models on latitude and longitude are easy to fit and to interpret. However, the use of simple parametric models for characterizing spatially varying processes may lead to misspecification problems if the model is not appropriate. Recently, Meilán-Vila et al. (2019) proposed a goodness-of-fit test based on an L2-distance for assessing a parametric trend model with correlated errors, under random design, comparing a parametric and a nonparametric trend estimators. The present work aims to provide a detailed computational analysis of the behavior of this approach using different bootstrap algorithms for calibration, under a fixed-design geostatistical framework. Asymptotic results for the test are provided and an extensive simulation study, considering complexities that usually arise in geostatistics, is carried out to illustrate the performance of the proposal.
Consider measured positions of the paleomagnetic north pole over time. Each measured position may be viewed as a direction, expressed as a unit vector in three dimensions and incorporating some error. In this sequence, the true directions are expected to be close to one another at nearby times. A simple trend estimator that respects the geometry of the sphere is to compute a running average over the time-ordered observed direction vectors, then normalize these average vectors to unit length. This paper treats a considerably richer class of competing directional trend estimators that respect sp…
This paper deals with the consistency, a rate of convergence and the asymptotic distribution of a nonparametric estimator of the trend in the Skorokhod reflection problem defined by a fractional SDE and a Moreau sweeping process.
We propose a novel approach for density estimation called histogram trend filtering. Our estimator arises from looking at surrogate Poisson model for counts of observations in a partition of the support of the data. We begin by showing consistency for a variational estimator for this density estimation problem. We then study a discrete estimator that can be efficiently found via convex optimization. We show that the estimator enjoys strong statistical guarantees, yet is much more practical and computationally efficient than other estimators that enjoy similar guarantees. Finally, in our simula…
Let X be a second order random process indexed by a compact interval [0,T]. Assume that n independent realizations of X are observed on a fixed grid of p time points. Under mild regularity assumptions on the sample paths of X, we show the asymptotic normality of suitable nonparametric estimators of the trend function mu = EX in the space C([0,T]) as n, p go to infinity and, using Gaussian process theory, we derive approximate simultaneous confidence bands for mu.
In this paper we propose an autoregressive wild bootstrap method to construct confidence bands around a smooth deterministic trend. The bootstrap method is easy to implement and does not require any adjustments in the presence of missing data, which makes it particularly suitable for climatological applications. We establish the asymptotic validity of the bootstrap method for both pointwise and simultaneous confidence bands under general conditions, allowing for general patterns of missing data, serial dependence and heteroskedasticity. The finite sample properties of the method are studied in…
The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functional parameters by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where …
Suppose that $Y = ψ(X_1, \ldots, X_p)$, where $(X_1,\ldots, X_p)^\top$ are random inputs, $Y$ is the output, and $ψ(\cdot)$ is an unknown link function. The Sobol indices gauge the sensitivity of each $X$ against $Y$ by estimating the regression curve's variability between them. In this paper, we estimate these curves with a kernel-based method. The method allows to estimate the first order indices when the link between the independent and dependent variables is unknown. The kernel-based methods need a bandwidth to average the observations. For finite samples, the cross-validation method is fa…
In this paper, we are interested in the problem of smoothing parameter selection in nonparametric curve estimation under dependent errors. We focus on kernel estimation and the case when the errors form a general stationary sequence of martingale difference random variables where neither linearity assumption nor "all moments are finite" are required.We compare the behaviors of the smoothing bandwidths obtained by minimizing either the unknown average squared error, the theoretical mean average squared error, a Mallows-type criterion adapted to the dependent case and the family of criteria know…
Mixed-effect models are widely used for the analysis of correlated data such as longitudinal data and repeated measures. In this article, we study an approach to the nonparametric estimation of mixed-effect models. We consider models with parametric random effects and flexible fixed effects, and employ the penalized least squares method to estimate the models. The issue to be addressed is the selection of smoothing parameters through the generalized cross-validation method, which is shown to yield optimal smoothing for both real and latent random effects. Simulation studies are conducted to in…
Based on the novel concept of multivariate center-outward quantiles introduced recently in Chernozhukov et al. (2017) and Hallin et al. (2021), we are considering the problem of nonparametric multiple-output quantile regression. Our approach defines nested conditional center-outward quantile regression contours and regions with given conditional probability content irrespective of the underlying distribution; their graphs constitute nested center-outward quantile regression tubes. Empirical counterparts of these concepts are constructed, yielding interpretable empirical regions and contours wh…
This paper develops a method to construct uniform confidence bands for a nonparametric regression function where a predictor variable is subject to a measurement error. We allow for the distribution of the measurement error to be unknown, but assume the availability of validation data or repeated measurements on the latent predictor variable. The proposed confidence band builds on the deconvolution kernel estimation and a novel application of the multiplier bootstrap method. We establish asymptotic validity of the proposed confidence band. To our knowledge, this is the first paper to derive as…
We demonstrate the consistency of cross validation for comparing multiple density estimators using simple inequalities on the likelihood ratio. In nonparametric problems, the splitting of data does not require the domination of test data over the training/estimation data, contrary to Shao (1993). The result is complementary to that of Yang (2005) and Yang (2006).
Carnegie Mellon University（卡内基梅隆大学）；University of Rome（罗马大学）
We derive nonparametric confidence intervals for the eigenvalues of the Hessian at modes of a density estimate. This provides information about the strength and shape of modes and can also be used as a significance test. We use a data-splitting approach in which potential modes are identified using the first half of the data and inference is done with the second half of the data. To get valid confidence sets for the eigenvalues, we use a bootstrap based on an elementary-symmetric-polynomial (ESP) transformation. This leads to valid bootstrap confidence sets regardless of any multiplicities in …
We study quantile trend filtering, a recently proposed method for nonparametric quantile regression with the goal of generalizing existing risk bounds known for the usual trend filtering estimators which perform mean regression. We study both the penalized and the constrained version (of order $r \geq 1$) of univariate quantile trend filtering. Our results show that both the constrained and the penalized version (of order $r \geq 1$) attain the minimax rate up to log factors, when the $(r-1)$th discrete derivative of the true vector of quantiles belongs to the class of bounded variation signal…
A piecewise-deterministic Markov process is a stochastic process whose behavior is governed by an ordinary differential equation punctuated by random jumps occurring at random times. We focus on the nonparametric estimation problem of the jump rate for such a stochastic model observed within a long time interval under an ergodicity condition. We introduce an uncountable class (indexed by the deterministic flow) of recursive kernel estimates of the jump rate and we establish their strong pointwise consistency as well as their asymptotic normality. We propose to choose among this class the estim…
We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute $k$th order discrete derivatives over the input points. Perhaps not surprisingly, trend filtering estimates appear to have the structure of $k$th degree spline functions, with adaptively chosen knot points (we say ``appear'' here as trend filtering estimates are not really functions over continuous domains, and are only defined over…
This work is motivated by a biological experiment with a split-plot design, for the purpose of comparison of the changing patterns in seed weight from two treatment groups as subgroups in each of the two groups subject to increasing levels of stress. We formalize the question into a nonparametric two sample comparison problem for changes among the sub samples, which was analyzed using U-statistics. Zero inflated value were also considered in the construction of the U-statistics. The U-statistics were then used in a Chi-square type test statistics framework for hypothesis testing. Bootstrapped …
We present an approach to deep estimation of discrete conditional probability distributions. Such models have several applications, including generative modeling of audio, image, and video data. Our approach combines two main techniques: dyadic partitioning and graph-based smoothing of the discrete space. By recursively decomposing each dimension into a series of binary splits and smoothing over the resulting distribution using graph-based trend filtering, we impose a strict structure to the model and achieve much higher sample efficiency. We demonstrate the advantages of our model through a s…
Estimation of genewise variance arises from two important applications in microarray data analysis: selecting significantly differentially expressed genes and validation tests for normalization of microarray data. We approach the problem by introducing a two-way nonparametric model, which is an extension of the famous Neyman--Scott model and is applicable beyond microarray data. The problem itself poses interesting challenges because the number of nuisance parameters is proportional to the sample size and it is not obvious how the variance function can be estimated when measurements are correl…
Prohorov Metric-Based Nonparametric Estimation of the Distribution of Random Parameters in Abstract Parabolic Systems with Application to the Transdermal Transport of Alcohol
Motivated by global warming issues, we consider a time se- ries that consists of a nondecreasing trend observed with station- ary fluctuations, nonparametric estimation of the trend under monotonicity assumption is considered. The rescaled isotonic es- timators at an interior point are shown to converge to Chernoff's distribution under minimal conditions on the stationary errors. Since the isotonic estimators suffer from the spiking problem at the end point, two modifications are proposed. The estima- tion errors for both estimators of the boundary point are shown to have interesting limiting …
The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation,…
In this paper, we establish minimax optimal rates of convergence for prediction in a semi-functional linear model that consists of a functional component and a less smooth nonparametric component. Our results reveal that the smoother functional component can be learned with the minimax rate as if the nonparametric component were known. More specifically, a double-penalized least squares method is adopted to estimate both the functional and nonparametric components within the framework of reproducing kernel Hilbert spaces. By virtue of the representer theorem, an efficient algorithm that requir…
We consider nonparametric estimation of cure-rate based on mixture model under Case-1 interval censoring. We show that the nonparametric maximum-likelihood estimator (NPMLE) of cure-rate is non-unique as well as inconsistent, and propose two estimators based on the NPMLE of the distribution function under this censoring model. We present a cross-validation method for choosing a `cut-off' point needed for the estimators. The limiting distributions of the latter are obtained using extreme-value theory. Graphical illustration of the procedures based on simulated data are provided.
A Nonparametric, Mixed Effect, Maximum Likelihood Estimator for the Distribution of Random Parameters in Discrete-Time Abstract Parabolic Systems with Application to the Transdermal Transport of Alcohol
Singular spectrum analysis (SSA) is considered for decomposition of time series into identifiable components. The Basic SSA method is nonparametric and constructs an adaptive expansion based on singular value decomposition. The investigated modification is able to take into consideration a structure given in advance and therefore can be called semi-nonparametric. The approach called SSA with projection includes preliminary projections of rows and columns of the series trajectory matrix to given subspaces. One application of SSA with projection is the extraction of polynomial trends, e.g., a li…
Here, we address the problem of trend estimation for functional time series. Existing contributions either deal with detecting a functional trend or assuming a simple model. They consider neither the estimation of a general functional trend nor the analysis of functional time series with a functional trend component. Similarly to univariate time series, we propose an alternative methodology to analyze functional time series, taking into account a functional trend component. We propose to estimate the functional trend by using a tensor product surface that is easy to implement, to interpret, an…
The COVID-19 pandemic is one of the most pressing issues at present. A question which is particularly important for governments and policy makers is the following: Does the virus spread in the same way in different countries? Or are there significant differences in the development of the epidemic? In this paper, we devise new inference methods that allow to detect differences in the development of the COVID-19 epidemic across countries in a statistically rigorous way. In our empirical study, we use the methods to compare the outbreak patterns of the epidemic in a number of European countries.
The order of smoothness chosen in nonparametric estimation problems is critical. This choice balances the tradeoff between model parsimony and data overfitting. The most common approach used in this context is cross-validation. However, cross-validation is computationally time consuming and often precludes valid post-selection inference without further considerations. With this in mind, borrowing elements from the objective Bayesian variable selection literature, we propose an approach to select the degree of a polynomial basis. Although the method can be extended to most series-based smoother…
Ethane is the most abundant non-methane hydrocarbon in the Earth's atmosphere and an important precursor of tropospheric ozone through various chemical pathways. Ethane is also an indirect greenhouse gas (global warming potential), influencing the atmospheric lifetime of methane through the consumption of the hydroxyl radical (OH). Understanding the development of trends and identifying trend reversals in atmospheric ethane is therefore crucial. Our dataset consists of four series of daily ethane columns obtained from ground-based FTIR measurements. As many other decadal time series, our data …
This paper studies nonparametric regression with long memory (LRD) errors and predictors. First, we formulate general conditions which guarantee the standard rate of convergence for a nonparametric kernel estimator. Second, we calculate the Mean Integrated Squared Error (MISE). In particular, we show that LRD of errors may influence MISE. On the other hand, an estimator for a shape function is typically not influenced by LRD in errors. Finally, we investigate properties of a data-driven bandwidth choice. We show that Averaged Squared Error (ASE) is a good approximation of MISE, however, this i…
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
Laboratoire de Mathématiques d’Orsay（奥赛数学实验室）；Univ. Paris-Sud（巴黎南大学）；CNRS（法国国家科学研究中心）；Université Paris-Saclay（巴黎萨克雷大学）；Univ. Nice Sophia Antipolis（尼斯索菲亚·安提波利斯大学）
This paper studies V-fold cross-validation for model selection in least-squares density estimation. The goal is to provide theoretical grounds for choosing V in order to minimize the least-squares loss of the selected estimator. We first prove a non-asymptotic oracle inequality for V-fold cross-validation and its bias-corrected version (V-fold penalization). In particular, this result implies that V-fold penalization is asymptotically optimal in the nonparametric case. Then, we compute the variance of V-fold cross-validation and related criteria, as well as the variance of key quantities for m…
In this paper, we propose new nonparametric approach to network inference that may be viewed as a fusion of block sampling procedures for temporally and spatially dependent processes with the classical network methodology. We develop estimation and uncertainty quantification procedures for network mean degree using a "patchwork" sample and nonparametric bootstrap, under the assumption of unknown degree distribution. We investigate asymptotic properties of the proposed patchwork bootstrap procedure and present cross-validation methodology for selecting an optimal patch size. We validate the new…
Cross-validation is a well-known and widely used bandwidth selection method in nonparametric regression estimation. However, this technique has two remarkable drawbacks: (i) the large variability of the selected bandwidths, and (ii) the inability to provide results in a reasonable time for very large sample sizes. To overcome these problems, bagging cross-validation bandwidths are analyzed in this paper. This approach consists in computing the cross-validation bandwidths for a finite number of subsamples and then rescaling the averaged smoothing parameters to the original sample size. Under a …
We consider the problem of bandwidth selection by cross-validation from a sequential point of view in a nonparametric regression model. Having in mind that in applications one often aims at estimation, prediction and change detection simultaneously, we investigate that approach for sequential kernel smoothers in order to base these tasks on a single statistic. We provide uniform weak laws of large numbers and weak consistency results for the cross-validated bandwidth. Extensions to weakly dependent error terms are discussed as well. The errors may be α-mixing or L2-near epoch dependent, which …
Yale University（耶鲁大学）；University of Michigan（密歇根大学）
Information theoretic measures (e.g. the Kullback Liebler divergence and Shannon mutual information) have been used for exploring possibly nonlinear multivariate dependencies in high dimension. If these dependencies are assumed to follow a Markov factor graph model, this exploration process is called structure discovery. For discrete-valued samples, estimates of the information divergence over the parametric class of multinomial models lead to structure discovery methods whose mean squared error achieves parametric convergence rates as the sample size grows. However, a naive application of thi…
Measuring training variability from stochastic optimization using robust nonparametric testing
Rutgers, The State University of New Jersey（罗格斯大学（新泽西州立大学））；Pacific Northwest National Lab（太平洋西北国家实验室）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；ARPA-H（美国高级卫生研究项目局）
We analyze the performance of cross-validation (CV) in the density estimation framework with two purposes: (i) risk estimation and (ii) model selection. The main focus is given to the so-called leave-$p$-out CV procedure (Lpo), where $p$ denotes the cardinality of the test set. Closed-form expressions are settled for the Lpo estimator of the risk of projection estimators. These expressions provide a great improvement upon $V$-fold cross-validation in terms of variability and computational complexity. From a theoretical point of view, closed-form expressions also enable to study the Lpo perform…
Financial econometrics has become an increasingly popular research field. In this paper we review a few parametric and nonparametric models and methods used in this area. After introducing several widely used continuous-time and discrete-time models, we study in detail dependence structures of discrete samples, including Markovian property, hidden Markovian structure, contaminated observations, and random samples. We then discuss several popular parametric and nonparametric estimation methods. To avoid model mis-specification, model validation plays a key role in financial modeling. We discuss…
The Highly-Adaptive-Lasso(HAL)-TMLE is an efficient estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance parameters are finite. It relies on an initial estimator (HAL-MLE) of the nuisance parameters by minimizing the empirical risk over the parameter space under the constraint that sectional variation norm is bounded by a constant, where this constant can be selected with cross-validation. In the formulation of the HALMLE this sectional variation norm corresponds with the sum o…
COVID-19 incidence is analyzed at the provinces of some Spanish Communities during the period February-October, 2020. Two infinite-dimensional regression approaches are tested. The first one is implemented in the regression framework introduced in Ruiz-Medina, Miranda and Espejo (2019). Specifically, a bayesian framework is adopted in the estimation of the pure point spectrum of the temporal autocorrelation operator, characterizing the second-order structure of a surface sequence. The second approach is formulated in the context of spatial curve regression. A nonparametric estimator of the spe…
