We consider the problem of constructing nonparametric undirected graphical models for high-dimensional functional data. Most existing statistical methods in this context assume either a Gaussian distribution on the vertices or linear conditional means. In this article we provide a more flexible model which relaxes the linearity assumption by replacing it by an arbitrary additive form. The use of functional principal components offers an estimation strategy that uses a group lasso penalty to estimate the relevant edges of the graph. We establish statistical guarantees for the resulting estimators, which can be used to prove consistency if the dimension and the number of functional principal components diverge to infinity with the sample size. We also investigate the empirical performance of our method through simulation studies and a real data application.
We consider nonparametric estimation of the mean and covariance functions for functional/longitudinal data. Strong uniform convergence rates are developed for estimators that are local-linear smoothers. Our results are obtained in a unified framework in which the number of observations within each curve/cluster can be of any rate relative to the sample size. We show that the convergence rates for the procedures depend on both the number of sample curves and the number of observations on each curve. For sparse functional data, these rates are equivalent to the optimal rates in nonparametric reg…
Pairwise Markov Random Fields (MRFs) or undirected graphical models are parsimonious representations of joint probability distributions. Variables correspond to nodes of a graph, with edges between nodes corresponding to conditional dependencies. Unfortunately, likelihood-based learning and inference is hampered by the intractability of computing the normalizing constant. This paper considers an alternative scoring rule to the log-likelihood, which obviates the need to compute the normalizing constant of the distribution. We show that the rule is a quadratic function of the natural parameters.…
Rice University（莱斯大学）；Baylor College of Medicine（贝勒医学院）
In neuroscience, researchers seek to uncover the connectivity of neurons from large-scale neural recordings or imaging; often people employ graphical model selection and estimation techniques for this purpose. But, existing technologies can only record from a small subset of neurons leading to a challenging problem of graph selection in the presence of extensive latent variables. Chandrasekaran et al. (2012) proposed a convex program to address this problem that poses challenges from both a computational and statistical perspective. To solve this problem, we propose an incredibly simple soluti…
Hawkes (1971) introduced a powerful multivariate point process model of mutually exciting processes to explain causal structure in data. In this paper it is shown that the Granger causality structure of such processes is fully encoded in the corresponding link functions of the model. A new nonparametric estimator of the link functions based on a time-discretized version of the point process is introduced by using an infinite order autoregression. Consistency of the new estimator is derived. The estimator is applied to simulated data and to neural spike train data from the spinal dorsal horn of…
In recent years, Ising prior with the network information for the "in" or "out" binary random variable in Bayesian variable selections has received more and more attentions. In this paper, we discover that even without the informative prior a Bayesian variable selection problem itself can be considered as a complete graph and described by a Ising model with random interactions. There are many advantages of treating variable selection as a graphical model, such as it is easy to employ the single site updating as well as the cluster updating algorithm, suitable for problems with small sample siz…
We consider nonparametric estimation of a covariance function on the unit square, given a sample of discretely observed fragments of functional data. When each sample path is only observed on a subinterval of length $δ<1$, one has no statistical information on the unknown covariance outside a $δ$-band around the diagonal. The problem seems unidentifiable without parametric assumptions, but we show that nonparametric estimation is feasible under suitable smoothness and rank conditions on the unknown covariance. This remains true even when observation is discrete, and we give precise determinist…
Princeton University（普林斯顿大学）；The University of Chicago（芝加哥大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We consider the problem of precision matrix estimation where, due to extraneous confounding of the underlying precision matrix, the data are independent but not identically distributed. While such confounding occurs in many scientific problems, our approach is inspired by recent neuroscientific research suggesting that brain function, as measured using functional magnetic resonance imagine (fMRI), is susceptible to confounding by physiological noise such as breathing and subject motion. Following the scientific motivation, we propose a graphical model, which in turn motivates a joint nonparame…
We present a method of variable selection for the sparse generalized additive model. The method doesn't assume any specific functional form, and can select from a large number of candidates. It takes the form of incremental forward stagewise regression. Given no functional form is assumed, we devised an approach termed roughening to adjust the residuals in the iterations. In simulations, we show the new method is competitive against popular machine learning approaches. We also demonstrate its performance using some real datasets. The method is available as a part of the nlnet package on CRAN h…
This thesis studies high-dimensional, continuous-valued pairwise Markov Random Fields. We are particularly interested in approximating pairwise densities whose logarithm belongs to a Sobolev space. For this problem we propose the method of exponential series [Crain, 1974; Barron and Sheu, 1991], which approximates the log density by a finite- dimensional exponential family with the number of sufficient statistics increasing with the sample size. We consider two approaches to estimating these models. The first is regularized maximum likelihood. This involves optimizing the sum of the log-likeli…
This article investigates nonparametric estimation of variance functions for functional data when the mean function is unknown. We obtain asymptotic results for the kernel estimator based on squared residuals. Similar to the finite dimensional case, our asymptotic result shows the smoothness of the unknown mean function has an effect on the rate of convergence. Our simulaton studies demonstrate that estimator based on residuals performs much better than that based on conditional second moment of the responses.
We address the estimation of quantiles from heavy-tailed distributions when functional covariate information is available and in the case where the order of the quantile converges to one as the sample size increases. Such "extreme" quantiles can be located in the range of the data or near and even beyond the boundary of the sample, depending on the convergence rate of their order to one. Nonparametric estimators of these functional extreme quantiles are introduced, their asymptotic distributions are established and their finite sample behavior is investigated.
We consider nonparametric regression in the context of functional data, that is, when a random sample of functions is observed on a fine grid. We obtain a functional asymptotic normality result allowing to build simultaneous confidence bands (SCB) for various estimation and inference tasks. Two applications to a SCB procedure for the regression function and to a goodness-of-fit test for curvilinear regression models are proposed. The first one has improved accuracy upon the other available methods while the second can detect local departures from a parametric shape, as opposed to the usual goo…
This paper is devoted to the study of large deviation behaviors in the setting of the estimation of the regression function on functional data. A large deviation principle is stated for a process Zn, defined below, allowing to derive a pointwise large deviation principle for the Nadaraya-Watson-type l-indexed regression function estimator as a by-product. Moreover, a uniform over VC-classes Cherno? type large deviation result is stated for the deviation of the l-indexed regression estimator.
We introduce the concept of pattern graphs--directed acyclic graphs representing how response patterns are associated. A pattern graph represents an identifying restriction that is nonparametrically identified/saturated and is often a missing not at random restriction. We introduce a selection model and a pattern mixture model formulations using the pattern graphs and show that they are equivalent. A pattern graph leads to an inverse probability weighting estimator as well as an imputation-based estimator. We also study the semi-parametric efficiency theory and derive a multiply-robust estimat…
We consider identification and inference about mean functionals of observed covariates and an outcome variable subject to nonignorable missingness. By leveraging a shadow variable, we establish a necessary and sufficient condition for identification of the mean functional even if the full data distribution is not identified. We further characterize a necessary condition for $\sqrt{n}$-estimability of the mean functional. This condition naturally strengthens the identifying condition, and it requires the existence of a function as a solution to a representer equation that connects the shadow va…
In this paper we prove large deviations principles for the Nadaraya-Watson estimator of the regression of a real-valued variable with a functional covariate. Under suitable conditions, we show pointwise and uniform large deviations theorems with good rate functions.
In this study, we develop an asymptotic theory of nonparametric regression for a locally stationary functional time series. First, we introduce the notion of a locally stationary functional time series (LSFTS) that takes values in a semi-metric space. Then, we propose a nonparametric model for LSFTS with a regression function that changes smoothly over time. We establish the uniform convergence rates of a class of kernel estimators, the Nadaraya-Watson (NW) estimator of the regression function, and a central limit theorem of the NW estimator.
We provide general adaptive upper bounds for estimating nonparametric functionals based on second order U-statistics arising from finite dimensional approximation of the infinite dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate.
We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition.
Predicting scalar outcomes using functional predictors is a classic problem in functional data analysis. In many applications, however, only specific locations or time-points of the functional predictors have an impact on the outcome. Such ``points of impact'' are typically unknown and have to be estimated in addition to estimating the usual model components. We show that our points of impact estimator enjoys a super-consistent convergence rate and does not require knowledge or pre-estimates of the unknown model components. This remarkable result facilitates the subsequent estimation of the re…
We consider nonparametric functional regression when both predictors and responses are functions. More specifically, we let $(X_1,Y_1),...,(X_n,Y_n)$ be random elements in $\mathcal{F}\times\mathcal{H}$ where $\mathcal{F}$ is a semi-metric space and $\mathcal{H}$ is a separable Hilbert space. Based on a recently introduced notion of weak dependence for functional data, we showed the almost sure convergence rates of both the Nadaraya-Watson estimator and the nearest neighbor estimator, in a unified manner. Several factors, including functional nature of the responses, the assumptions on the fun…
We study nonparametric clustering of smooth random curves on the basis of the L2 gradient flow associated to a pseudo-density functional and we show that the clustering is well-defined both at the population and at the sample level. We provide an algorithm to mark significant local modes, which are associated to informative sample clusters, and we derive its consistency properties. Our theory is developed under weak assumptions, which essentially reduce to the integrability of the random curves, and does not require to project the random curves on a finite-dimensional subspace. However, if the…
We consider the problem of predicting a real random variable from a functional explanatory variable. The problem is attacked by mean of nonparametric kernel approach which has been recently adapted to this functional context. We derive theoretical results by giving a deep asymptotic study of the behaviour of the estimate, including mean squared convergence (with rates and precise evaluation of the constant terms) as well as asymptotic distribution. Practical use of these results are relying on the ability to estimate these constants. Some perspectives in this direction are discussed including …
The covariance structure of multivariate functional data can be highly complex, especially if the multivariate dimension is large, making extensions of statistical methods for standard multivariate data to the functional data setting challenging. For example, Gaussian graphical models have recently been extended to the setting of multivariate functional data by applying multivariate methods to the coefficients of truncated basis expansions. However, a key difficulty compared to multivariate data is that the covariance operator is compact, and thus not invertible. The methodology in this paper …
Consider a Poisson point process with unknown support boundary curve $g$, which forms a prototype of an irregular statistical model. We address the problem of estimating non-linear functionals of the form $\int Φ(g(x))\,dx$. Following a nonparametric maximum-likelihood approach, we construct an estimator which is UMVU over Hölder balls and achieves the (local) minimax rate of convergence. These results hold under weak assumptions on $Φ$ which are satisfied for $Φ(u)=|u|^p$, $p\ge 1$. As an application, we consider the problem of estimating the $L^p$-norm and derive the minimax separation rates…
In this paper, a nonparametric estimator is proposed for estimating the L1-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional L1-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built…
Motivated by modeling and analysis of mass-spectrometry data, a semi- and nonparametric model is proposed that consists of a linear parametric component for individual location and scale and a nonparametric regression function for the common shape. A multi-step approach is developed that simultaneously estimates the parametric components and the nonparametric function. Under certain regularity conditions, it is shown that the resulting estimators is consistent and asymptotic normal for the parametric part and achieve the optimal rate of convergence for the nonparametric part when the bandwidth…
The main purpose is to estimate the regression function of a real random variable with functional explanatory variable by using a recursive nonparametric kernel approach. The mean square error and the almost sure convergence of a family of recursive kernel estimates of the regression function are derived. These results are established with rates and precise evaluation of the constant terms. Also, a central limit theorem for this class of estimators is established. The method is evaluated on simulations and real data set studies.
This paper examines the problem of nonparametric testing for the no-effect of a random covariate (or predictor) on a functional response. This means testing whether the conditional expectation of the response given the covariate is almost surely zero or not, without imposing any model relating response and covariate. The covariate could be univariate, multivariate or functional. Our test statistic is a quadratic form involving univariate nearest neighbor smoothing and the asymptotic critical values are given by the standard normal law. When the covariate is multidimensional or functional, a pr…
For nonparametric regression with one-sided errors and a boundary curve model for Poisson point processes we consider the problem of efficient estimation for linear functionals. The minimax optimal rate is obtained by an unbiased estimation method which nevertheless depends on a Hölder condition or monotonicity assumption for the underlying regression or boundary function. We first construct a simple blockwise estimator and then build up a nonparametric maximum-likelihood approach for exponential noise variables and the point process model. In that approach also non-asymptotic efficiency is ob…
We establish minimax convergence rates for classification of functional data and for nonparametric regression with functional design variables. The optimal rates are of logarithmic type under smoothness constraints on the functional density and the regression mapping, respectively. These asymptotic properties are attainable by conventional kernel procedures. The bandwidth selector does not require knowledge of the smoothness level of the target mapping. In this work, the functional data are considered as realisations of random variables which take their values in a general Polish metric space.…
In this paper, we consider a functional linear regression model, where both the covariate and the response variable are functional random variables. We address the problem of optimal nonparametric estimation of the conditional expectation operator in this model. A collection of projection estimators over finite dimensional subspaces is first introduce. We provide a non-asymptotic bias-variance decomposition for the Mean Square Prediction error in the case where these subspaces are generated by the (empirical) PCA functional basis. The automatic trade-off is realized thanks to a model selection…
Functional graphical models explore dependence relationships of random processes. This is achieved through estimating the precision matrix of the coefficients from the Karhunen-Loeve expansion. This paper deals with the problem of estimating functional graphs that consist of the same random processes and share some of the dependence structure. By estimating a single graph we would be shrouding the uniqueness of different sub groups within the data. By estimating a different graph for each sub group we would be dividing our sample size. Instead, we propose a method that allows joint estimation …
We consider statistical models where functional data are artificially contaminated by independent Wiener processes in order to satisfy privacy constraints. We show that the corrupted observations have a Wiener density which determines the distribution of the original functional random variables, masked near the origin, uniquely, and we construct a nonparametric estimator of that density. We derive an upper bound for its mean integrated squared error which has a polynomial convergence rate, and we establish an asymptotic lower bound on the minimax convergence rates which is close to the rate at…
We propose NonStGM, a general nonparametric graphical modeling framework for studying dynamic associations among the components of a nonstationary multivariate time series. It builds on the framework of Gaussian Graphical Models (GGM) and stationary time series Gaussian Graphical model (StGM), and complements existing works on parametric graphical models based on change point vector autoregressions (VAR). Analogous to StGM, the proposed framework captures conditional noncorrelations (both intertemporal and contemporaneous) in the form of an undirected graph. In addition, to describe the more n…
Fully nonparametric methods for regression from functional data have poor accuracy from a statistical viewpoint, reflecting the fact that their convergence rates are slower than nonparametric rates for the estimation of high-dimensional functions. This difficulty has led to an emphasis on the so-called functional linear model, which is much more flexible than common linear models in finite dimension, but nevertheless imposes structural constraints on the relationship between predictors and responses. Recent advances have extended the linear approach by using it in conjunction with link functio…
This paper studies the problem of nonparametric testing for the effect of a random functional covariate on a real-valued error term. The covariate takes values in $L^2[0,1]$, the Hilbert space of the square-integrable real-valued functions on the unit interval. The error term could be directly observed as a response or \emph{estimated} from a functional parametric model, like for instance the functional linear regression. Our test is based on the remark that checking the no-effect of the functional covariate is equivalent to checking the nullity of the conditional expectation of the error term…
We introduce a nonparametric graphical model for discrete node variables based on additive conditional independence. Additive conditional independence is a three way statistical relation that shares similar properties with conditional independence by satisfying the semi-graphoid axioms. Based on this relation we build an additive graphical model for discrete variables that does not suffer from the restriction of a parametric model such as the Ising model. We develop an estimator of the new graphical model via the penalized estimation of the discrete version of the additive precision operator a…
This paper considers adaptive, minimax estimation of a quadratic functional in a nonparametric instrumental variables (NPIV) model, which is an important problem in optimal estimation of a nonlinear functional of an ill-posed inverse regression with an unknown operator. We first show that a leave-one-out, sieve NPIV estimator of the quadratic functional can attain a convergence rate that coincides with the lower bound previously derived in Chen and Christensen [2018]. The minimax rate is achieved by the optimal choice of the sieve dimension (a key tuning parameter) that depends on the smoothne…
We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish convergence rates of the posterior measure of the regression function and its derivatives, which are both minimax optimal up to a logarithmic factor for functions in certain classes. Our calculation shows that the rate-optimal estimation of the regression function and its derivatives share the same choice of hyperparameter, indicating that the Bayes procedure remarkably adapts to …
We consider the problem of estimating the value l(ϕ) of a linear functional, where the structural function ϕ models a nonparametric relationship in presence of instrumental variables. We propose a plug-in estimator which is based on a dimension reduction technique and additional thresholding. It is shown that this estimator is consistent and can attain the minimax optimal rate of convergence under additional regularity conditions. This, however, requires an optimal choice of the dimension parameter m depending on certain characteristics of the structural function ϕ and the joint distribution o…
The paper considers probability distribution, density, conditional distribution and density and conditional moments as well as their kernel estimators in spaces of generalized functions. This approach does not require restrictions on classes of distributions common in nonparametric estimation. Density in usual function spaces is not well-posed; this paper establishes existence and well-posedness of the generalized density function. It also demonstrates root-n convergence of the kernel density estimator in the space of generalized functions. It is shown that the usual kernel estimator of the co…
We consider the problem of estimating the slope parameter in functional linear regression, where scalar responses Y1,...,Yn are modeled in dependence of second order stationary random functions X1,...,Xn. An orthogonal series estimator of the functional slope parameter with additional thresholding in the Fourier domain is proposed and its performance is measured with respect to a wide range of weighted risks covering as examples the mean squared prediction error and the mean integrated squared error for derivative estimation. In this paper the minimax optimal rate of convergence of the estimat…
We consider the problem of estimating the value of a linear functional in nonparametric instrumental regression, where in the presence of an instrument W a response Y is modeled in dependence of an endogenous explanatory variable Z. The proposed estimator is based on dimension reduction and additional thresholding. The minimax optimal rate of convergence of the estimator is derived assuming that the structural function and the representer of the linear functional belong to some ellipsoids which are in a certain sense linked to the conditional expectation operator of Z given W. We illustrate th…
