Spatial interaction patterns such as segregation and association can be tested using nearest neighbor contingency tables (NNCTs). We introduce new cell-specific (or pairwise) and overall segregation tests and determine their asymptotic distributions. In particular, we demonstrate that cell-specific tests enjoy asymptotic normality, while overall tests have chi-square distributions asymptotically. We also perform an extensive Monte Carlo simulation study to compare the finite sample performance of the tests in terms of empirical size and power. In addition to the cell-specific tests as post-hoc tests for overall tests, we discuss one-class-versus-rest type of NNCT-tests after an overall test yields significant interaction. We also introduce the concepts of total, strong, and partial segregation/association to label levels of these patterns. We compare these new tests with the existing NNCT-tests in literature with simulations as well and illustrate the NNCT-tests on an ecological data set.
E-variables are tools for retaining type-I error guarantee with optional stopping. We extend E-variables for sequential two-sample tests to general null hypotheses and anytime-valid confidence sequences. We provide implementations for estimating risk difference, relative risk and odds-ratios in contingency tables.
There are two popular general approaches for the analysis and visualization of a contingency table and a compositional data set: Correspondence analysis (CA) and log ratio analysis (LRA). LRA includes two independently well developed methods: association models and compositional data analysis. The application of either CA or LRA to a contingency table or to compositional data set includes a preprocessing centering step. In CA the centering step is multiplicative, while in LRA it is log bi-additive. A preprocessed matrix is double-centered, so it is a residuel matrix; which implies that it affe…
In this work we define log-linear models to compare several square contingency tables under the quasi-independence or the quasi-symmetry model, and the relevant Markov bases are theoretically characterized. Through Markov bases, an exact test to evaluate if two or more tables fit a common model is introduced. Two real-data examples illustrate the use of these models in different fields of applications.
The analysis of contingency tables is a powerful statistical tool used in experiments with categorical variables. This study improves parts of the theory underlying the use of contingency tables. Specifically, the linkage disequilibrium parameter as a measure of two-way interactions applied to three-way tables makes it possible to quantify Simpsons paradox by a simple formula. With tests on three-way interactions, there is only one that determines whether the partial interactions of all variables agree or whether there is at least one variable whose partial interactions disagree. To date, ther…
We present a new approach for random sampling of contingency tables of any size and constraints based on a recently introduced $\textit{probabilistic divide-and-conquer}$ technique. A simple exact sampling algorithm is presented for $2\times n$ tables, as well as a generalization where each entry of the table has a specified marginal distribution.
Exact tests greatly improve the analysis of contingency tables when marginals are low. For instance, researchers often use Fisher's exact test, which is conditional, or Barnard's test, which is unconditional but needs to deal with a nuisance parameter. Here, we describe the m-test, an exact, unconditional test for the study of d x m binomial contingency tables. When comparing binomial trials, the m-test is related to Barnard's exact test. However, the nuisance parameter is integrated over all its possible values, instead of maximized or otherwise estimated. According to Monte Carlo simulations…
We obtain sharp asymptotic estimates on the number of $n \times n$ contingency tables with two linear margins $Cn$ and $BCn$. The results imply a second order phase transition on the number of such contingency tables, with a critical value at \ts $B_{c}:=1 + \sqrt{1+1/C}$. As a consequence, for \ts $B>B_{c}$, we prove that the classical \emph{independence heuristic} leads to a large undercounting.
Contingency tables are useful objects in statistics for representing 2-way data. With fixed row and column sums, and a total of $n$ entries, contingency tables correspond to parabolic double cosets of $S_n$. The uniform distribution on $S_n$ induces the Fisher-Yates distribution, classical for its use in the chi-squared test for independence. A Markov chain on $S_n$ can then induce a random process on the space of contingency tables through the double cosets correspondence. The random transpositions Markov chain on $S_n$ induces a natural `swap' Markov chain on the space of contingency tables;…
It is an important inferential problem to test no association between two binary variables based on data. Tests based on the sample odds ratio are commonly used. We bring in a competing test based on the Pearson correlation coefficient. In particular, the Odds ratio does not extend to higher order contingency tables, whereas Pearson correlation does. It is important to understand how Pearson correlation stacks against the odds ratio in 2x2 tables. Another measure of association is the canonical correlation. In this paper, we examine how competitive Pearson correlation is vis-à-vis odds ratio i…
Hypothesis testing in contingency tables is usually based on asymptotic results, thereby restricting its proper use to large samples. To study these tests in small samples, we consider the likelihood ratio test and define an accurate index, the P-value, for the celebrated hypotheses of homogeneity, independence, and Hardy-Weinberg equilibrium. The aim is to understand the use of the asymptotic results of the frequentist Likelihood Ratio Test and the Bayesian FBST -- Full Bayesian Significance Test -- under small-sample scenarios. The proposed exact P-value is used as a benchmark to understand …
We demonstrate a novel approach for the random sampling of Latin squares of order~$n$ via probabilistic divide-and-conquer. The algorithm divides the entries of the table modulo powers of $2$, and samples a corresponding binary contingency table at each level. The sampling distribution is based on the Boltzmann sampling heuristic, along with probabilistic divide-and-conquer.
We assessed several agreement coefficients applied in 2x2 contingency tables, which are commonly applied in research due to dicotomization by the conditions of the subjects (e.g., male or female) or by conveniency of the classification (e.g., traditional thresholds leading to separations in healthy or diseased, exposed or non-exposed, etc.). More extreme table configurations (e.g., high agreement between raters) are also usual, but some of the coefficients have problems with imbalanced tables. Here, we not only studied some especific estimators, but also developed a general method to the study…
We consider the asymptotic distribution of a cell in a 2 x ... x 2 contingency table as the fixed marginal totals tend to infinity. The asymptotic order of the cell variance is derived and a useful diagnostic is given for determining whether the cell has a Poisson limit or a Gaussian limit. There are three forms of Poisson convergence. The exact form is shown to be determined by the growth rates of the two smallest marginal totals. The results are generalized to contingency tables with arbitrary sizes and are further complemented with concrete examples.
We consider the lattice, $\mathcal{L}$, of all subsets of a multidimensional contingency table and establish the properties of monotonicity and supermodularity for the marginalization function, $n(\cdot)$, on $\mathcal{L}$. We derive from the supermodularity of $n(\cdot)$ some generalized Fréchet inequalities complementing and extending inequalities of Dobra and Fienberg. Further, we construct new monotonic and supermodular functions from $n(\cdot)$, and we remark on the connection between supermodularity and some correlation inequalities for probability distributions on lattices. We also appl…
Visualization and interpretation of contingency tables by correspondence analysis (CA), as developed by Benzecri, has a rich structure based on Euclidean geometry. However, it is a well established fact that, often CA is very sensitive to sparse contingency tables, where we caracterize sparsity as the existence of relatively high-valued counts, rare observations discussed by Rao (1995), and zero-block structure emphasized by Novak and Bar-Hen (2005) and Greenacre (2013). In this paper, we aim to emphasize the important roles played by L1 and L2 geometries. This will be done by comparing the ma…
Log-linear modeling is a popular method for the analysis of contingency table data. When the table is sparse, and the data falls on a proper face $F$ of the convex support, there are consequences on model inference and model selection. Knowledge of the cells determining $F$ is crucial to mitigating these effects. We introduce the R package (R Core Team (2016)) eMLEloglin for determining $F$ and passing that information on to the glm package to fit the model properly.
Methods for selecting loglinear models were among Steve Fienberg's research interests since the start of his long and fruitful career. After we dwell upon the string of papers focusing on loglinear models that can be partly attributed to Steve's contributions and influential ideas, we develop a new algorithm for selecting graphical loglinear models that is suitable for analyzing hyper-sparse contingency tables. We show how multi-way contingency tables can be used to represent patterns of human mobility. We analyze a dataset of geolocated tweets from South Africa that comprises 46 million latit…
