Classical two-sample permutation tests for equality of distributions have exact size in finite samples, but they fail to control size for testing equality of parameters that summarize each distribution. This paper proposes permutation tests for equality of parameters that are estimated at root-$n$ or slower rates. Our general framework applies to both parametric and nonparametric models, with two samples or one sample split into two subsamples. Our tests have correct size asymptotically while preserving exact size in finite samples when distributions are equal. They have no loss in local asymptotic power compared to tests that use asymptotic critical values. We propose confidence sets with correct coverage in large samples that also have exact coverage in finite samples if distributions are equal up to a transformation. We apply our theory to four commonly-used hypothesis tests of nonparametric functions evaluated at a point. Lastly, simulations show good finite sample properties, and two empirical examples illustrate our tests in practice.
Measures of local and global spatial association are key tools for exploratory spatial data analysis. Many such measures exist including Moran's $I$, Geary's $C$, and the Getis-Ord $G$ and $G^*$ statistics. A parametric approach to testing for significance relies on strong assumptions, which are often not met by real world data. Alternatively, the most popular nonparametric approach, the permutation test, imposes a large computational burden especially for massive graphical networks. Hence, we propose a computation-free approach to nonparametric permutation testing for local and global measure…
University of Washington（华盛顿大学）；Google Inc（谷歌公司）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Fred Hutchinson Cancer Research Center（弗雷德·哈钦森癌症研究中心）
This paper investigates the theoretical and empirical performance of Fisher-Pitman-type permutation tests for assessing the equality of unknown Poisson mixture distributions. Building on nonparametric maximum likelihood estimators (NPMLEs) of the mixing distribution, these tests are theoretically shown to be able to adapt to complicated unspecified structures of count data and also consistent against their corresponding ANOVA-type alternatives; the latter is a result in parallel to classic claims made by Robinson (Robinson, 1973). The studied methods are then applied to a single-cell RNA-seq d…
We describe and examine a test for a general class of shape constraints, such as constraints on the signs of derivatives, U-(S-)shape, symmetry, quasi-convexity, log-convexity, $r$-convexity, among others, in a nonparametric framework using partial sums empirical processes. We show that, after a suitable transformation, its asymptotic distribution is a functional of the standard Brownian motion, so that critical values are available. However, due to the possible poor approximation of the asymptotic critical values to the finite sample ones, we also describe a valid bootstrap algorithm.
We develop a nonparametric extension of the sequential generalized likelihood ratio (GLR) test and corresponding time-uniform confidence sequences for the mean of a univariate distribution. By utilizing a geometric interpretation of the GLR statistic, we derive a simple analytic upper bound on the probability that it exceeds any prespecified boundary; these are intractable to approximate via simulations due to infinite horizon of the tests and the composite nonparametric nulls under consideration. Using time-uniform boundary-crossing inequalities, we carry out a unified nonasymptotic analysis …
We point out necessary and sufficient conditions of uniform consistency of nonparametric sets of alternatives for widespread nonparametric tests. Nonparametric sets of alternatives can be defined both in terms of distribution function and in terms of density (or signals in the problem of signal detection in Gaussian white noise). In this part of paper such conditions are provided for $χ^2-$tests with increasing number of cells, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients.
Regional frequency analysis is an important tool to properly estimate hydrological characteristics at ungauged or partially gauged sites in order to prevent hydrological disasters. The delineation of homogeneous groups of sites is an important first step in order to transfer information and obtain accurate quantile estimates at the target site. The Hosking-Wallis homogeneity test is usually used to test the homogeneity of the selected sites. Despite its usefulness and good power, it presents some drawbacks including the subjective choice of a parametric distribution for the data and a poorly j…
A common challenge in nonparametric inference is its high computational complexity when data volume is large. In this paper, we develop computationally efficient nonparametric testing by employing a random projection strategy. In the specific kernel ridge regression setup, a simple distance-based test statistic is proposed. Notably, we derive the minimum number of random projections that is sufficient for achieving testing optimality in terms of the minimax rate. An adaptive testing procedure is further established without prior knowledge of regularity. One technical contribution is to establi…
Hypothesis tests based on linear models are widely accepted by organizations that regulate clinical trials. These tests are derived using strong assumptions about the data-generating process so that the resulting inference can be based on parametric distributions. Because these methods are well understood and robust, they are sometimes applied to data that depart from assumptions, such as ordinal integer scores. Permutation tests are a nonparametric alternative that require minimal assumptions which are often guaranteed by the randomization that was conducted. We compare analysis of covariance…
We discuss nonparametric tests for parametric specifications of regression quantiles. The test is based on the comparison of parametric and nonparametric fits of these quantiles. The nonparametric fit is a Nadaraya-Watson quantile smoothing estimator. An asymptotic treatment of the test statistic requires the development of new mathematical arguments. An approach that makes only use of plugging in a Bahadur expansion of the nonparametric estimator is not satisfactory. It requires too strong conditions on the dimension and the choice of the bandwidth. Our alternative mathematical approach requi…
In this paper, we are concerned with nonparametric estimation of the multivariate regression function in the presence of right censored data. More precisely, we propose a statistic that is shown to be asymptotically normally distributed under the additive assumption, and that could be used to test for additivity in the censored regression setting.
The Bayesian nonparametric inference and Dirichlet process are popular tools in statistical methodologies. In this paper, we employ the Dirichlet process in hypothesis testing to propose a Bayesian nonparametric chi-squared goodness-of-fit test. In our Bayesian nonparametric approach, we consider the Dirichlet process as the prior for the distribution of data and carry out the test based on the Kullback-Leibler distance between the updated Dirichlet process and the hypothesized distribution F0. We prove that this distance asymptotically converges to the same chi-squared distribution as the chi…
Nonparametric estimation of a mixing distribution based on data coming from a mixture model is a challenging problem. Beyond estimation, there is interest in uncertainty quantification, e.g., confidence intervals for features of the mixing distribution. This paper focuses on estimation via the predictive recursion algorithm, and here we take advantage of this estimator's seemingly undesirable dependence on the data ordering to obtain a permutation-based approximation of the sampling distribution which can be used to quantify uncertainty. Theoretical and numerical results confirm that the propo…
We investigate hypothesis testing in nonparametric additive models estimated using simplified smooth backfitting (Huang and Yu, Journal of Computational and Graphical Statistics, \textbf{28(2)}, 386--400, 2019). Simplified smooth backfitting achieves oracle properties under regularity conditions and provides closed-form expressions of the estimators that are useful for deriving asymptotic properties. We develop a generalized likelihood ratio (GLR) and a loss function (LF) based testing framework for inference. Under the null hypothesis, both the GLR and LF tests have asymptotically rescaled ch…
The class of composite likelihood functions provides a flexible and powerful toolkit to carry out approximate inference for complex statistical models when the full likelihood is either impossible to specify or unfeasible to compute. However, the strenght of the composite likelihood approach is dimmed when considering hypothesis testing about a multidimensional parameter because the finite sample behavior of likelihood ratio, Wald, and score-type test statistics is tied to the Godambe information matrix. Consequently inaccurate estimates of the Godambe information translate in inaccurate p-val…
In this paper we introduce the idea of partially sorting data to design nonparametric tests. This approach gives rise to tests that are sensitive to both the order and the underlying distribution of the data. We focus in particular on a test that uses the bubble sort algorithm to partially sort the data. We show that a function of the data, referred to as the empirical bubble sort curve, converges uniformly to a limiting curve. We define a goodness-of-fit test based on the distance between the empirical curve and its limit. The asymptotic distribution of the test statistic is a generalization …
We consider nonparametric testing in a non-asymptotic framework. Our statistical guarantees are exact in the sense that Type I and II errors are controlled for any finite sample size. Meanwhile, one proposed test is shown to achieve minimax optimality in the asymptotic sense. An important consequence of this non-asymptotic theory is a new and practically useful formula for selecting the optimal smoothing parameter in nonparametric testing. The leading example in this paper is smoothing spline models under Gaussian errors. The results obtained therein can be further generalized to the kernel ri…
We present a general approach to constructing permutation tests that are both exact for the null hypothesis of equality of distributions and asymptotically correct for testing equality of parameters of distributions while allowing the distributions themselves to differ. These robust permutation tests transform a given test statistic by a consistent estimator of its limiting distribution function before enumerating its permutation distribution. This transformation, known as prepivoting, aligns the unconditional limiting distribution for the test statistic with the probability limit of its permu…
Motivated by the prevalence of high dimensional low sample size datasets in modern statistical applications, we propose a general nonparametric framework, Direction-Projection-Permutation (DiProPerm), for testing high dimensional hypotheses. The method is aimed at rigorous testing of whether lower dimensional visual differences are statistically significant. Theoretical analysis under the non-classical asymptotic regime of dimension going to infinity for fixed sample size reveals that certain natural variations of DiProPerm can have very different behaviors. An empirical power study both confi…
For the problems of nonparametric hypothesis testing we introduce the notion of maxisets and maxispace. We point out the maxisets of $χ^2-$tests, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients. For these tests we show that, if sequence of alternatives having given rates of convergence to hypothesis is consistent, then each altehrnative can be broken down into the sum of two parts: a function belonging to maxiset and orthogonal function. Sequence of functions belonging to maxiset is con…
Consider a Poisson point process with unknown support boundary curve $g$, which forms a prototype of an irregular statistical model. We address the problem of estimating non-linear functionals of the form $\int Φ(g(x))\,dx$. Following a nonparametric maximum-likelihood approach, we construct an estimator which is UMVU over Hölder balls and achieves the (local) minimax rate of convergence. These results hold under weak assumptions on $Φ$ which are satisfied for $Φ(u)=|u|^p$, $p\ge 1$. As an application, we consider the problem of estimating the $L^p$-norm and derive the minimax separation rates…
We provide new asymptotic theory for kernel density estimators, when these are applied to autoregressive processes exhibiting moderate deviations from a unit root. This fills a gap in the existing literature, which has to date considered only nearly integrated and stationary autoregressive processes. These results have applications to nonparametric predictive regression models. In particular, we show that the null rejection probability of a nonparametric t test is controlled uniformly in the degree of persistence of the regressor. This provides a rigorous justification for the validity of the …
In this paper an autoregressive time series model with conditional heteroscedasticity is considered, where both conditional mean and conditional variance function are modeled nonparametrically. A test for the model assumption of independence of innovations from past time series values is suggested. The test is based on an weighted $L^2$-distance of empirical characteristic functions. The asymptotic distribution under the null hypothesis of independence is derived and consistency against fixed alternatives is shown. A smooth autoregressive residual bootstrap procedure is suggested and its perfo…
In recent years, Bayesian nonparametric statistics has gathered extraordinary attention. Nonetheless, a relatively little amount of work has been expended on Bayesian nonparametric hypothesis testing. In this paper, a novel Bayesian nonparametric approach to the two-sample problem is established. Precisely, given two samples $\mathbf{X}=X_1,\ldots,X_{m_1}$ $\overset {i.i.d.} \sim F$ and $\mathbf{Y}=Y_1,\ldots,Y_{m_2} \overset {i.i.d.} \sim G$, with $F$ and $G$ being unknown continuous cumulative distribution functions, we wish to test the null hypothesis $\mathcal{H}_0:~F=G$. The method is bas…
In this paper we propose a new test for the hypothesis of a constant coefficient of variation in the common nonparametric regression model. The test is based on an estimate of the $L^2$-distance between the square of the regression function and variance function. We prove asymptotic normality of a standardized estimate of this distance under the null hypothesis and fixed alternatives and the finite sample properties of a corresponding bootstrap test are investigated by means of a simulation study. The results are applicable to stationary processes with the common mixing conditions and are used…
We introduce new test statistic to test the independence of two multi-dimensional random variables. Based on the $L_1$-distance and the historgram density estimation method, the test is compared via Bahadur relative efficiency to several tests available in the literature. It arises that our test reaches better performances than a number of usual tests among whom we cite the Kolmogorov-Smirnov test. Beforehand, large deviation result is stated for the associated statistic. The local asymptotic optimality relative to the test is also studied.
Nonparametric tests via kernel embedding of distributions have witnessed a great deal of practical successes in recent years. However, statistical properties of these tests are largely unknown beyond consistency against a fixed alternative. To fill in this void, we study here the asymptotic properties of goodness-of-fit, homogeneity and independence tests using Gaussian kernels, arguably the most popular and successful among such tests. Our results provide theoretical justifications for this common practice by showing that tests using Gaussian kernel with an appropriately chosen scaling parame…
We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions. We propose a penalized likelihood ratio test for such interaction testing and show that the test statistic is asymptotically chi-square distributed under the null hypothesis. Furthermore, we derive a sharp minimax testing rate based on the Bernstein width for nonparametric two-s…
We consider the detection problem of a two-dimensional function from noisy observations of its integrals over lines. We study both rate and sharp asymptotics for the error probabilities in the minimax setup. By construction, the derived tests are non-adaptive. We also construct a minimax rate-optimal adaptive test of rather simple structure.
Given a nonparametric Hidden Markov Model (HMM) with two states, the question of constructing efficient multiple testing procedures is considered, treating one of the states as an unknown null hypothesis. A procedure is introduced, based on nonparametric empirical Bayes ideas, that controls the False Discovery Rate (FDR) at a user--specified level. Guarantees on power are also provided, in the form of a control of the true positive rate. One of the key steps in the construction requires supremum--norm convergence of preliminary estimators of the emission densities of the HMM. We provide the ex…
This paper considers the problem of multi-sample nonparametric comparison of counting processes with panel count data, which arise naturally when recurrent events are considered. Such data frequently occur in medical follow-up studies and reliability experiments, for example. For the problem considered, we construct two new classes of nonparametric test statistics based on the accumulated weighted differences between the rates of increase of the estimated mean functions of the counting processes over observation times, wherein the nonparametric maximum likelihood approach is used to estimate t…
We propose a nonparametric procedure to test for changes in correlation matrices at an unknown point in time. The new test requires only mild assumptions on the serial dependence structure and has considerable power in finite samples. We derive the asymptotic distribution under the null hypothesis of no change as well as local power results and apply the test to stock returns.
We consider the problem of testing whether two finite-dimensional random dot product graphs have generating latent positions that are independently drawn from the same distribution, or distributions that are related via scaling or projection. We propose a test statistic that is a kernel-based function of the adjacency spectral embedding for each graph. We obtain a limiting distribution for our test statistic under the null and we show that our test procedure is consistent across a broad range of alternatives.
University of California, San Diego（加州大学圣地亚哥分校）；University of California, Berkeley（加州大学伯克利分校）
We propose a new algorithmic framework for sequential hypothesis testing with i.i.d. data, which includes A/B testing, nonparametric two-sample testing, and independence testing as special cases. It is novel in several ways: (a) it takes linear time and constant space to compute on the fly, (b) it has the same power guarantee as a non-sequential version of the test with the same computational constraints up to a small factor, and (c) it accesses only as many samples as are required - its stopping time adapts to the unknown difficulty of the problem. All our test statistics are constructed to b…
Hypothesis tests are a crucial statistical tool for data mining and are the workhorse of scientific research in many fields. Here we study differentially private tests of independence between a categorical and a continuous variable. We take as our starting point traditional nonparametric tests, which require no distributional assumption (e.g., normality) about the data distribution. We present private analogues of the Kruskal-Wallis, Mann-Whitney, and Wilcoxon signed-rank tests, as well as the parametric one-sample t-test. These tests use novel test statistics developed specifically for the pr…
The problem of curve registration appears in many different areas of applications ranging from neuroscience to road traffic modeling. In the present work, we propose a nonparametric testing framework in which we develop a generalized likelihood ratio test to perform curve registration. We first prove that, under the null hypothesis, the resulting test statistic is asymptotically distributed as a chi-squared random variable. This result, often referred to as Wilks' phenomenon, provides a natural threshold for the test of a prescribed asymptotic significance level and a natural measure of lack-o…
We address the issue of lack-of-fit testing for a parametric quantile regression. We propose a simple test that involves one-dimensional kernel smoothing, so that the rate at which it detects local alternatives is independent of the number of covariates. The test has asymptotically gaussian critical values, and wild bootstrap can be applied to obtain more accurate ones in small samples. Our procedure appears to be competitive with existing ones in simulations. We illustrate the usefulness of our test on birthweight data.
For $χ^2-$tests with increasing number of cells, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients, we find necessary and sufficient conditions of consistency and inconsistency for sequences of alternatives having a given rate of convergence to hypothesis in $\mathbb{L}_2$-norm. We provide transparent interpretations of these conditions allowing to understand the structure of such consistent sequences. For problem of signal detection in Gaussian white noise we show that, if set of alterna…
When permutation methods are used in practice, often a limited number of random permutations are used to decrease the computational burden. However, most theoretical literature assumes that the whole permutation group is used, and methods based on random permutations tend to be seen as approximate. There exists a very limited amount of literature on exact testing with random permutations and only recently a thorough proof of exactness was given. In this paper we provide an alternative proof, viewing the test as a "conditional Monte Carlo test" as it has been called in the literature. We also p…
In a functional setting, we propose two test statistics to highlight the Poisson nature of a Cox process when n copies of the process are available. Our approach involves a comparison of the empirical mean and the empirical variance of the functional data and can be seen as an extended version of a classical overdispersion test for counting data. The limiting distributions of our statistics are derived using a functional central limit theorem for c`adl`ag martingales. We also study the asymptotic power of our tests under some local alternatives. Our procedure is easily implementable and does n…
This paper is about two related decision theoretic problems, nonparametric two-sample testing and independence testing. There is a belief that two recently proposed solutions, based on kernels and distances between pairs of points, behave well in high-dimensional settings. We identify different sources of misconception that give rise to the above belief. Specifically, we differentiate the hardness of estimation of test statistics from the hardness of testing whether these statistics are zero or not, and explicitly discuss a notion of "fair" alternative hypotheses for these problems as dimensio…
A dimension reduction-based adaptive-to-model test is proposed for significance of a subset of covariates in the context of a nonparametric regression model. Unlike existing local smoothing significance tests, the new test behaves like a local smoothing test as if the number of covariates were just that under the null hypothesis and it can detect local alternatives distinct from the null at the rate that is only related to the number of covariates under the null hypothesis. Thus, the curse of dimensionality is largely alleviated when nonparametric estimation is inevitably required. In the case…
The problem of testing for the parametric form of the conditional variance is considered in a fully nonparametric regression model. A test statistic based on a weighted $L_2$-distance between the empirical characteristic functions of residuals constructed under the null hypothesis and under the alternative is proposed and studied theoretically. The null asymptotic distribution of the test statistic is obtained and employed to approximate the critical values. Finite sample properties of the proposed test are numerically investigated in several Monte Carlo experiments. The developed results assu…
Generalized likelihood ratio statistics have been proposed in Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153-193] as a generally applicable method for testing nonparametric hypotheses about nonparametric functions. The likelihood ratio statistics are constructed based on the assumption that the distributions of stochastic errors are in a certain parametric family. We extend their work to the case where the error distribution is completely unspecified via newly proposed sieve empirical likelihood ratio (SELR) tests. The approach is also applied to test conditional estimating equations on the…
Nonparametric generalized likelihood ratio test is popularly used for model checking for regressions. However, there are two issues that may be the barriers for its powerfulness. First, the bias term in its liming null distribution causes the test not to well control type I error and thus Monte Carlo approximation for critical value determination is required. Second, it severely suffers from the curse of dimensionality due to the use of multivariate nonparametric function estimation. The purpose of this paper is thus two-fold: a bias-correction is suggested to this test and a dimension reducti…
The nonparametric test for change-point detection proposed by Gombay and Horváth is revisited and extended in the broader setting of empirical process theory. The resulting testing procedure for potentially multivariate observations is based on a sequential generalization of the functional multiplier central limit theorem and on modifications of Gombay and Horváth's seminal approach that appears to improve the finite-sample behavior of the tests. A large number of candidate test statistics based on processes indexed by lower-left orthants and half-spaces are considered and their performance is…
Mutual information is a well-known tool to measure the mutual dependence between variables. In this paper, a Bayesian nonparametric estimation of mutual information is established by means of the Dirichlet process and the $k$-nearest neighbor distance. As a direct outcome of the estimation, an easy-to-implement test of independence is introduced through the relative belief ratio. Several theoretical properties of the approach are presented. The procedure is investigated through various examples where the results are compared to its frequentist counterpart and demonstrate a good performance.
We develop a group of robust, nonparametric hypothesis tests which detect differences between the covariance operators of several populations of functional data. These tests, called FKWC tests, are based on functional data depth ranks. These tests work well even when the data is heavy tailed, which is shown both in simulation and theoretically. These tests offer several other benefits, they have a simple distribution under the null hypothesis, they are computationally cheap and they possess transformation invariance properties. We show that under general alternative hypotheses these tests are …
We propose a flexible and identifiable version of the two-groups model, motivated by hierarchical Bayes considerations, that features an empirical null and a semiparametric mixture model for the non-null cases. We use a computationally efficient predictive recursion marginal likelihood procedure to estimate the model parameters, even the nonparametric mixing distribution. This leads to a nonparametric empirical Bayes testing procedure, which we call PRtest, based on thresholding the estimated local false discovery rates. Simulations and real-data examples demonstrate that, compared to existing…
In the context of data that lie on the simplex, we investigate use of empirical and exponential empirical likelihood, and Hotelling and James statistics, to test the null hypothesis of equal population means based on two independent samples. We perform an extensive numerical study using data simulated from various distributions on the simplex. The results, taken together with practical considerations regarding implementation, support the use of bootstrap-calibrated James statistic.
