The differential-geometric structure of the set of positive densities on a given measure space has raised the interest of many mathematicians after the discovery by C.R. Rao of the geometric meaning of the Fisher information. Most of the research is focused on parametric statistical models. In series of papers by author and coworkers a particular version of the nonparametric case has been discussed. It consists of a minimalistic structure modeled according the theory of exponential families: given a reference density other densities are represented by the centered log likelihood which is an element of an Orlicz space. This mappings give a system of charts of a Banach manifold. It has been observed that, while the construction is natural, the practical applicability is limited by the technical difficulty to deal with such a class of Banach spaces. It has been suggested recently to replace the exponential function with other functions with similar behavior but polynomial growth at infinity in order to obtain more tractable Banach spaces, e.g. Hilbert spaces. We give first a review of our theory with special emphasis on the specific issues of the infinite dimensional setting. In a second part we discuss two specific topics, differential equations and the metric connection. The position of this line of research with respect to other approaches is briefly discussed.
This article provides an exposition of recent methodologies for nonparametric analysis of digital observations on images and other non-Euclidean objects. Fréchet means of distributions on metric spaces, such as manifolds and stratified spaces, have played an important role in this endeavor. Apart from theoretical issues of uniqueness of the Fréchet minimizer and the asymptotic distribution of the sample Fréchet mean under uniqueness, applications to image analysis are highlighted. In addition, nonparametric Bayes theory is brought to bear on the problems of density estimation and classificatio…
Surface integrals on density level sets often appear in asymptotic results in nonparametric level set estimation (such as for confidence regions and bandwidth selection). Also surface integrals can be used to describe the shape of level sets (using such as Willmore energy and Minkowski functionals), and link geometry (curvature) and topology (Euler characteristic) of level sets through the Gauss-Bonnet theorem. We consider three estimators of surface integrals on density level sets, one as a direct plug-in estimator, and the other two based on different neighborhoods of level sets. We allow th…
A central goal of causal inference is to detect and estimate the treatment effects of a given treatment or intervention on an outcome variable of interest, where a member known as the heterogeneous treatment effect (HTE) is of growing popularity in recent practical applications such as the personalized medicine. In this paper, we model the HTE as a smooth nonparametric difference between two less smooth baseline functions, and determine the tight statistical limits of the nonparametric HTE estimation as a function of the covariate geometry. In particular, a two-stage nearest-neighbor-based est…
This article considers a nonparametric method for detecting change points in non-stationary time series. The proposed method will divide the time series into several segments so that between two adjacent segments, the normalized spectral density functions are different. The theory is based on the assumption that within each segment, time series is a linear process, which means that our method works not only for classic time series models, e.g., causal and invertible ARMA process, but also preserves good performance for non-invertible moving average process. We show that our estimations for cha…
This paper provides a mathematical framework for estimation of the service time distribution and the expected service time of an infinite-server queueing system with a nonhomogeneous Poisson arrival process, in the case of partial information, where only the number of busy servers are observed over time. The problem is reduced to a statistical deconvolution problem, which is solved by using Laplace transform techniques and kernels for regularization. Upper bounds on the mean squared error of the proposed estimators are derived. Some concrete simulation experiments are performed to illustrate h…
Mutual information is a well-known tool to measure the mutual dependence between variables. In this paper, a Bayesian nonparametric estimation of mutual information is established by means of the Dirichlet process and the $k$-nearest neighbor distance. As a direct outcome of the estimation, an easy-to-implement test of independence is introduced through the relative belief ratio. Several theoretical properties of the approach are presented. The procedure is investigated through various examples where the results are compared to its frequentist counterpart and demonstrate a good performance.
We investigate existence and properties of discrete mixture representations $P_θ=\sum_{i\in E} w_θ(i) \, Q_i$ for a given family $P_θ$, $θ\inΘ$, of probability measures. The noncentral chi-squared distributions provide a classical example. We obtain existence results and results about geometric and statistical aspects of the problem, the latter including loss of Fisher information, Rao-Blackwellization, asymptotic efficiency and nonparametric maximum likelihood estimation of the mixing probabilities.
We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and marginal entropies facilitates the use of recently-developed efficient entropy estimators derived from nearest neighbour distances. The proposed critical values, which may be obtained from simulation (in the case where one marginal is known) or resampling, guarantee that the test has nominal size, and we provide local power analyses, uniformly over classes of de…
Department of Statistics, University of Washington（华盛顿大学统计系）；Department of Statistics, University of Chicago（芝加哥大学统计系）
The Gaussian-smoothed optimal transport (GOT) framework, pioneered in Goldfeld et al. (2020) and followed up by a series of subsequent papers, has quickly caught attention among researchers in statistics, machine learning, information theory, and related fields. One key observation made therein is that, by adapting to the GOT framework instead of its unsmoothed counterpart, the curse of dimensionality for using the empirical measure to approximate the true data generating distribution can be lifted. The current paper shows that a related observation applies to the estimation of nonparametric m…
Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines, pattern recognition, data mining and tests of independence. The main goal of this paper is to provide an efficient estimator of the mutual information based on the approach of Al Labadi et. al. (2021). The estimator is explored through various examples and is compared to its frequentist counterpart due to Berrett et al. (2019). The results show the good per…
This paper studies and critically discusses the construction of nonparametric confidence regions for density level sets. Methodologies based on both vertical variation and horizontal variation are considered. The investigations provide theoretical insight into the behavior of these confidence regions via large sample theory. We also discuss the geometric relationships underlying the construction of horizontal and vertical methods, and how finite sample performance of these confidence regions is influenced by geometric or topological aspects. These discussions are supported by numerical studies…
University of Electronic Science and Technology of China（电子科技大学）；Princeton University（普林斯顿大学）
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed a clipped estimator, is proposed. Superior upper bounds on the rates of convergence can be shown for the new estimator compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, u…
We study the multivariate nonparametric change point detection problem, where the data are a sequence of independent $p$-dimensional random vectors whose distributions are piecewise-constant with Lipschitz densities changing at unknown times, called change points. We quantify the size of the distributional change at any change point with the supremum norm of the difference between the corresponding densities. We are concerned with the localization task of estimating the positions of the change points. In our analysis, we allow for the model parameters to vary with the total number of time poin…
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in $d$-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accuracy of every NPMLE. Our results imply, in particular, that every NPMLE achieves near parametric risk (up to logarithmic multiplicative factors) when the true density is a discrete Gaussian mixture without any prior information on the number of mixture components. NPMLEs can natural…
Data can be assumed to be continuous functions defined on an infinite-dimensional space for many phenomena. However, the infinite-dimensional data might be driven by a small number of latent variables. Hence, factor models are relevant for functional data. In this paper, we study functional factor models for time-dependent functional data. We propose nonparametric estimators under stationary and nonstationary processes. We obtain estimators that consider the time-dependence property. Specifically, we use the information contained in the covariances at different lags. We show that the proposed …
University of Pennsylvania（宾夕法尼亚大学）；Yale University（耶鲁大学）
This paper studies the problem of nonparametric estimation of a smooth function with data distributed across multiple machines. We assume an independent sample from a white noise model is collected at each machine, and an estimator of the underlying true function needs to be constructed at a central machine. We place limits on the number of bits that each machine can use to transmit information to the central machine. Our results give both asymptotic lower bounds and matching upper bounds on the statistical risk under various settings. We identify three regimes, depending on the relationship a…
Nonparametric estimation of the conditional expectation $E(Y | U)$ of an outcome $Y$ given a covariate vector $U$ is of primary importance in many statistical applications such as prediction and personalized medicine. In some problems, there is an additional auxiliary variable $Z$ in the training dataset used to construct estimators, but $Z$ is not available for future prediction or selecting patient treatment in personalized medicine. For example, in the training dataset longitudinal outcomes are observed, but only the last outcome $Y$ is concerned in the future prediction or analysis. The lo…
Many panel data have the latent subgroup effect on individuals, and it is important to correctly identify these groups since the efficiency of resulting estimators can be improved significantly by pooling the information of individuals within each group. However, the currently assumed parametric and semiparametric relationship between the response and predictors may be misspecified, which leads to a wrong grouping result, and the nonparametric approach hence can be considered to avoid such mistakes. Moreover, the response may depend on predictors in different ways at various quantile levels, a…
With the rapid growth of neuroimaging technologies, a great effort has been dedicated recently to investigate the dynamic changes in brain activity. Examples include time course calcium imaging and dynamic brain functional connectivity. In this paper, we propose a novel nonparametric matrix response regression model to characterize the nonlinear association between 2D image outcomes and predictors such as time and patient information. Our estimation procedure can be formulated as a nuclear norm regularization problem, which can capture the underlying low-rank structure of the dynamic 2D images…
This paper provides a systematic approach to semiparametric identification that is based on statistical information as a measure of its "quality". Identification can be regular or irregular, depending on whether the Fisher information for the parameter is positive or zero, respectively. I first characterize these cases in models with densities linear in a nonparametric parameter. I then introduce a novel "generalized Fisher information". If positive, it implies (possibly irregular) identification when other conditions hold. If zero, it implies impossibility results on rates of estimation. Thre…
Nonparametric Prior Learning in Differential Equation Modeling
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））；Center for Statistical Science, Peking University（北京大学统计科学中心）
The information detection of complex systems from data is currently undergoing a revolution, driven by the emergence of big data and machine learning methodology. Discovering governing equations and quantifying dynamical properties of complex systems are among central challenges. In this work, we devise a nonparametric approach to learn the relative entropy rate from observations of stochastic differential equations with different drift functions.The estimator corresponding to the relative entropy rate then is presented via the Gaussian process kernel theory. Meanwhile, this approach enables t…
Distance correlation is a recent extension of Pearson's correlation, that characterises general statistical independence between Euclidean-space-valued random variables, not only linear relations. This review delves into how and when distance correlation can be extended to metric spaces, combining the information that is available in the literature with some original remarks and proofs, in a way that is comprehensible for any mathematical statistician.
Università degli Studi di Milano（米兰大学）；INRIA（法国国家信息与自动化研究所）；École Normale Supérieure（巴黎高等师范学院）；Università degli Studi dell’Insubria（因苏布里亚大学）；Université Toulouse III - Paul Sabatier（图卢兹第三大学（保罗·萨巴蒂埃大学））
We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedback model motivated by second-price auctions, we obtain algorithms for Lipschitz and semi-Lipschitz losses with regret bounds improving on the known bounds for standard bandit feedback. Our analysis combines novel results for contextual second-price auctions with a novel algorith…
