One of the most popular class of tests for independence between two random variables is the general class of rank statistics which are invariant under permutations. This class contains Spearman's coefficient of rank correlation statistic, Fisher-Yates statistic, weighted Mann statistic and others. Under the null hypothesis of independence these test statistics have a permutation distribution that usually the normal asymptotic theory used to approximate the p-values for these tests. In this note we suggest using a saddlepoint approach that almost exact and need no extensive simulation calculations to calculate the p-value of such class of tests.
We propose a new class of models for random permutations, which we call log-linear models, by the analogy with log-linear models used in the analysis of contingency tables. As a special case, we study the family of all Luce-decomposable distributions, and the family of those random permutations, for which the distribution of both the permutation and its inverse is Luce-decomposable. We show that these latter models can be described by conditional independence relations. We calculate the number of free parameters in these models, and describe an iterative algorithm for maximum likelihood estima…
In this paper, we investigate local permutation tests for testing conditional independence between two random vectors $X$ and $Y$ given $Z$. The local permutation test determines the significance of a test statistic by locally shuffling samples which share similar values of the conditioning variables $Z$, and it forms a natural extension of the usual permutation approach for unconditional independence testing. Despite its simplicity and empirical support, the theoretical underpinnings of the local permutation test remain unclear. Motivated by this gap, this paper aims to establish theoretical …
We study the problem of independence testing given independent and identically distributed pairs taking values in a $σ$-finite, separable measure space. Defining a natural measure of dependence $D(f)$ as the squared $L^2$-distance between a joint density $f$ and the product of its marginals, we first show that there is no valid test of independence that is uniformly consistent against alternatives of the form $\{f: D(f) \geq ρ^2 \}$. We therefore restrict attention to alternatives that impose additional Sobolev-type smoothness constraints, and define a permutation test based on a basis expansi…
The Hilbert--Schmidt Independence Criterion (HSIC) is a popular measure of the dependency between two random variables. The statistic dHSIC is an extension of HSIC that can be used to test joint independence of $d$ random variables. Such hypothesis testing for (joint) independence is often done using a permutation test, which compares the observed data with randomly permuted datasets. The main contribution of this work is proving that the power of such independence tests converges to 1 as the sample size converges to infinity. This answers a question that was asked in (Pfister, 2018) Additiona…
In nonparametric independence testing, we observe i.i.d.\ data $\{(X_i,Y_i)\}_{i=1}^n$, where $X \in \mathcal{X}, Y \in \mathcal{Y}$ lie in any general spaces, and we wish to test the null that $X$ is independent of $Y$. Modern test statistics such as the kernel Hilbert-Schmidt Independence Criterion (HSIC) and Distance Covariance (dCov) have intractable null distributions due to the degeneracy of the underlying U-statistics. Thus, in practice, one often resorts to using permutation testing, which provides a nonasymptotic guarantee at the expense of recalculating the quadratic-time statistics …
This paper investigates the problem of testing independence of two random vectors of general dimensions. For this, we give for the first time a distribution-free consistent test. Our approach combines distance covariance with the center-outward ranks and signs developed in Hallin (2017). In technical terms, the proposed test is consistent and distribution-free in the family of multivariate distributions with nonvanishing (Lebesgue) probability densities. Exploiting the (degenerate) U-statistic structure of the distance covariance and the combinatorial nature of Hallin's center-outward ranks an…
We investigate the problem of testing whether $d$ random variables, which may or may not be continuous, are jointly (or mutually) independent. Our method builds on ideas of the two variable Hilbert-Schmidt independence criterion (HSIC) but allows for an arbitrary number of variables. We embed the $d$-dimensional joint distribution and the product of the marginals into a reproducing kernel Hilbert space and define the $d$-variable Hilbert-Schmidt independence criterion (dHSIC) as the squared distance between the embeddings. In the population case, the value of dHSIC is zero if and only if the $…
Mutual information is a well-known tool to measure the mutual dependence between variables. In this paper, a Bayesian nonparametric estimation of mutual information is established by means of the Dirichlet process and the $k$-nearest neighbor distance. As a direct outcome of the estimation, an easy-to-implement test of independence is introduced through the relative belief ratio. Several theoretical properties of the approach are presented. The procedure is investigated through various examples where the results are compared to its frequentist counterpart and demonstrate a good performance.
We propose an estimator of the Hilbert-Schmidt Independence Criterion obtained from an appropriate modification of the usual estimator. We then get asymptotic normality of this estimator both under independence hypothesis and under the alternative hypothesis. A new test for independence of random variables valued into metric spaces is then introduced, and a simulation study that allows to compare the proposed test to an existing one is provided
In this work we define log-linear models to compare several square contingency tables under the quasi-independence or the quasi-symmetry model, and the relevant Markov bases are theoretically characterized. Through Markov bases, an exact test to evaluate if two or more tables fit a common model is introduced. Two real-data examples illustrate the use of these models in different fields of applications.
Testing (conditional) independence of multivariate random variables is a task central to statistical inference and modelling in general - though unfortunately one for which to date there does not exist a practicable workflow. State-of-art workflows suffer from the need for heuristic or subjective manual choices, high computational complexity, or strong parametric assumptions. We address these problems by establishing a theoretical link between multivariate/conditional independence testing, and model comparison in the multivariate predictive modelling aka supervised learning task. This link all…
We propose an independence test for random variables valued into metric spaces by using a test statistic obtained from appropriately centering and rescaling the squared Hilbert-Schmidt norm of the usual empirical estimator of normalized cross-covariance operator. We then get asymptotic normality of this statistic under independence hypothesis, so leading to a new test for independence of functional random variables. A simulation study that allows to compare the proposed test to existing ones is provided.
Model selection for Gaussian concentration graph is based on multiple testing of pairwise conditional independence. In practical applications partial correlation tests are widely used. However it is not known whether partial correlation test is uniformly most powerful for pairwise conditional independence testing. This question is answered in the paper. Uniformly most powerful unbiased test of Neymann structure is obtained. It turns out, that this test can be reduced to usual partial correlation test. It implies that partial correlation test is uniformly most powerful unbiased one.
As a crucial problem in statistics is to decide whether additional variables are needed in a regression model. We propose a new multivariate test to investigate the conditional mean independence of Y given X conditioning on some known effect Z, i.e., E(Y|X, Z) = E(Y|Z). Assuming that E(Y|Z) and Z are linearly related, we reformulate an equivalent notion of conditional mean independence through transformation, which is approximated in practice. We apply the martingale difference divergence (Shao and Zhang, 2014) to measure conditional mean dependence, and show that the estimation error from app…
We introduce new test statistic to test the independence of two multi-dimensional random variables. Based on the $L_1$-distance and the historgram density estimation method, the test is compared via Bahadur relative efficiency to several tests available in the literature. It arises that our test reaches better performances than a number of usual tests among whom we cite the Kolmogorov-Smirnov test. Beforehand, large deviation result is stated for the associated statistic. The local asymptotic optimality relative to the test is also studied.
Robust Independence Tests with Finite Sample Guarantees for Synchronous Stochastic Linear Systems
SZTAKI: Institute for Computer Science and Control, Eötvös Loránd Research Network（厄特沃什·罗兰研究网络SZTAKI：计算机科学与控制研究所）；Eötvös Loránd University (ELTE)（厄特沃什·罗兰大学（ELTE））
In this paper, we propose a procedure to test the independence of bivariate censored data, which is generic and applicable to any censoring types in the literature. To test the hypothesis, we consider a rank-based statistic, Kendall's tau statistic. The censored data defines a restricted permutation space of all possible ranks of the observations. We propose the statistic, the average of Kendall's tau over the ranks in the restricted permutation space. To evaluate the statistic and its reference distribution, we develop a Markov chain Monte Carlo (MCMC) procedure to obtain uniform samples on t…
When permutation methods are used in practice, often a limited number of random permutations are used to decrease the computational burden. However, most theoretical literature assumes that the whole permutation group is used, and methods based on random permutations tend to be seen as approximate. There exists a very limited amount of literature on exact testing with random permutations and only recently a thorough proof of exactness was given. In this paper we provide an alternative proof, viewing the test as a "conditional Monte Carlo test" as it has been called in the literature. We also p…
We present a general approach to constructing permutation tests that are both exact for the null hypothesis of equality of distributions and asymptotically correct for testing equality of parameters of distributions while allowing the distributions themselves to differ. These robust permutation tests transform a given test statistic by a consistent estimator of its limiting distribution function before enumerating its permutation distribution. This transformation, known as prepivoting, aligns the unconditional limiting distribution for the test statistic with the probability limit of its permu…
For a set of dependent random variables, without stationary or the strong mixing assumptions, we derive the asymptotic independence between their sums and maxima. Then we apply this result to high-dimensional testing problems, where we combine the sum-type and max-type tests and propose a novel test procedure for the one-sample mean test, the two-sample mean test and the regression coefficient test in high-dimensional setting. Based on the asymptotic independence between sums and maxima, the asymptotic distributions of test statistics are established. Simulation studies show that our proposed …
CREST（法国 Crest 研究中心）；ENSAE（法国国家统计与经济管理学院）；Facebook AI Research（脸书人工智能研究院）；Université Paris-Dauphine（巴黎多芬大学）；Technion（以色列理工学院）
We propose a new conditional dependence measure and a statistical test for conditional independence. The measure is based on the difference between analytic kernel embeddings of two well-suited distributions evaluated at a finite set of locations. We obtain its asymptotic distribution under the null hypothesis of conditional independence and design a consistent statistical test from it. We conduct a series of experiments showing that our new test outperforms state-of-the-art methods both in terms of type-I and type-II errors even in the high dimensional setting.
We present the $U$-Statistic Permutation (USP) test of independence in the context of discrete data displayed in a contingency table. Either Pearson's chi-squared test of independence, or the $G$-test, are typically used for this task, but we argue that these tests have serious deficiencies, both in terms of their inability to control the size of the test, and their power properties. By contrast, the USP test is guaranteed to control the size of the test at the nominal level for all sample sizes, has no issues with small (or zero) cell counts, and is able to detect distributions that violate i…
In the present article, we propose jackknife empirical likelihood (JEL) ratio test for testing the independence of time to failure and cause of failure in competing risks data. We use U-statistic theory to derive the JEL ratio test. The asymptotic distribution of the test statistic is shown to be chi-square distribution with one degree of freedom. A Monte Carlo simulation study is carried out to assess the finite sample behaviour of the proposed test. The performance of proposed JEL test is compared with the test given in Dewan et al. (2004). Finally we illustrate our test procedure using vari…
Competing risks data with discrete lifetime comes up in practice. However, only limited literature exists for such data. In this paper, we propose a non-parametric test based on U-statistics for testing independence of time to failure and cause of failure of competing risks data when the lifetime is a discrete random variable. Asymptotic distribution of the proposed test statistic is derived. An extensive Monte Carlo simulation study is conducted to assess the finite sample performance of the proposed test. The flexibility of the testing procedure is illustrated using real data sets on oral ca…
Permutation testing in linear models, where the number of nuisance coefficients is smaller than the sample size, is a well-studied topic. The common approach of such tests is to permute residuals after regressing on the nuisance covariates. Permutation-based tests are valuable in particular because they can be highly robust to violations of the standard linear model, such as non-normality and heteroscedasticity. Moreover, in some cases they can be combined with existing, powerful permutation-based multiple testing methods. Here, we propose permutation tests for models where the number of nuisa…
A new test of independence between random elements is presented in this article. The test is based on a functional of the Cramér-von Mises type, which is applied to a $U$-process that is defined from the recurrence rates. Theorems of asymptotic distribution under $H_{0},$ and consistency under a wide class of alternatives are obtained. The results under contiguous alternatives are also shown. The test has a very good behaviour under several alternatives, which shows that in many cases there is clearly larger power when compared to other tests that are widely used in literature. In addition, th…
We study a class of distributed hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error rate subject to a (constant) upper bound $ε$ on the Type I error rate, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings.
Permutation tests are a distribution free way of performing hypothesis tests. These tests rely on the condition that the observed data are exchangeable among the groups being tested under the null hypothesis. This assumption is easily satisfied for data obtained from a simple random sample or a controlled study after simple adjustments to the data, but there is no general method for adjusting survey data collected using a complex sample design to allow for permutation tests. In this article, we propose a general method for performing a pseudo- permutation test that accounts for the complex sam…
We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution of its margins, are derived through the dual representation of phi-divergences. The asymptotic properties of the proposed estimates are established, including consistency, asymptotic distributions and large deviations principle. The obtained tests of independence are compared via their relative asymptotic Bahadur efficiency and numerical simulations. It foll…
This paper develops a model-free sequential test for conditional independence. The proposed test allows researchers to analyze an incoming i.i.d. data stream with any arbitrary dependency structure, and safely conclude whether a feature is conditionally associated with the response under study. We allow the processing of data points online, as soon as they arrive, and stop data acquisition once significant results are detected, rigorously controlling the type-I error rate. Our test can work with any sophisticated machine learning algorithm to enhance data efficiency to the extent possible. The…
The partial copula provides a method for describing the dependence between two random variables $X$ and $Y$ conditional on a third random vector $Z$ in terms of nonparametric residuals $U_1$ and $U_2$. This paper develops a nonparametric test for conditional independence by combining the partial copula with a quantile regression based method for estimating the nonparametric residuals. We consider a test statistic based on generalized correlation between $U_1$ and $U_2$ and derive its large sample properties under consistency assumptions on the quantile regression procedure. We demonstrate thro…
A distributed binary hypothesis testing (HT) problem over a noisy (discrete and memoryless) channel studied previously by the authors is investigated from the perspective of the strong converse property. It was shown by Ahlswede and Csiszár that a strong converse holds in the above setting when the channel is rate-limited and noiseless. Motivated by this observation, we show that the strong converse continues to hold in the noisy channel setting for a special case of HT known as testing against independence (TAI). The proof utilizes the blowing up lemma and the recent change of measure techniq…
Hypothesis testing is a useful statistical tool in determining whether a given model should be rejected based on a sample from the population. Sample data may contain sensitive information about individuals, such as medical information. Thus it is important to design statistical tests that guarantee the privacy of subjects in the data. In this work, we study hypothesis testing subject to differential privacy, specifically chi-squared tests for goodness of fit for multinomial data and independence between two categorical variables. We propose new tests for goodness of fit and independence testi…
The permutation test is an often used test procedure in brain imaging. Unfortunately, generating every possible permutation for large-scale brain image datasets such as HCP and ADNI with hundreds images is not practical. Many previous attempts at speeding up the permutation test rely on various approximation strategies such as estimating the tail distribution with known parametric distributions. In this study, we show how to rapidly accelerate the permutation test without any type of approximate strategies by exploiting the underlying algebraic structure of the permutation group. The method is…
Conditional independence tests (CI tests) have received special attention lately in Machine Learning and Computational Intelligence related literature as an important indicator of the relationship among the variables used by their models. In the field of Probabilistic Graphical Models (PGM)--which includes Bayesian Networks (BN) models--CI tests are especially important for the task of learning the PGM structure from data. In this paper, we propose the Full Bayesian Significance Test (FBST) for tests of conditional independence for discrete datasets. FBST is a powerful Bayesian test for precis…
Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including nonlinear associations and multivariate interactions. However, these approaches come with an at least quadratic computational cost in the number of observations, which can be prohibitive in many applications. Arguably, it is exactly in such large-scale datasets that capturing any type of dependence is of interest, so striking a favourable tradeoff between computati…
In 1948 Hoeffding devised a nonparametric test that detects dependence between two continuous random variables X and Y, based on the ranking of n paired samples (Xi,Yi). The computation of this commonly-used test statistic takes O(n log n) time. Hoeffding's test is consistent against any dependent probability density f(x,y), but can be fooled by other bivariate distributions with continuous margins. Variants of this test with full consistency have been considered by Blum, Kiefer, and Rosenblatt (1961), Yanagimoto (1970), Bergsma and Dassios (2010). The so far best known algorithms to compute t…
In this paper we describe the testing procedure for assessing the statistical significance of treatment effect under the experimental conditions of baseline imbalance across covariates and attrition from the survey, using the permutation tests proposed by Freedman and Lane (1983) and Romano and Wolf (2016). We discuss the testing procedure for these hypotheses based on a linear regression model and introduce the new Stata command [R] permtest for the implementation of the permutation test in Stata. Moreover, we investigate the finite-sample performance as well as the statistical validity of th…
Researchers in genetics and other life sciences commonly use permutation tests to evaluate differences between groups. Permutation tests have desirable properties, including exactness if data are exchangeable, and are applicable even when the distribution of the test statistic is analytically intractable. However, permutation tests can be computationally intensive. We propose both an asymptotic approximation and a resampling algorithm for quickly estimating small permutation p-values (e.g. $<10^{-6}$) for the difference and ratio of means in two-sample tests. Our methods are based on the distr…
