We study the random binary contingency tables with non-uniform margin. More precisely, for parameters $n,δ,B,C$, we consider $X=(X_{ij})$ with $X_{ij}\in \lbrace 0,1\rbrace$, the random binary contingency tables whose first $[n^δ]$ rows and columns have margin $[BCn]$ and the rest columns and rows have margin $[Cn]$. We study various asymptotic properties of $X$ as $n\to \infty$. This answers a question posted by Barvinok.
Let $C\geq 2$ be a positive integer. Consider the set of $n\times n$ non-negative integer matrices whose row sums and column sums are all equal to $Cn$ and let $X=(X_{ij})_{1\leq i,j\leq n}$ be uniformly distributed on this set. This $X$ is called the random contingency table with uniform margin. In this paper, we study various asymptotic properties of $X=(X_{ij})_{1\leq i,j\leq n}$ as $n\to\infty$.
We present a new approach for random sampling of contingency tables of any size and constraints based on a recently introduced $\textit{probabilistic divide-and-conquer}$ technique. A simple exact sampling algorithm is presented for $2\times n$ tables, as well as a generalization where each entry of the table has a specified marginal distribution.
We study a generalisation of Simpson reversal (also known as Simpson's paradox or the Yule-Simpson effect) to $2 \times 2 \times 2$ contingency tables and characterise the cases for which it can and cannot occur with two combinatorial-geometric lemmas. We also present a conjecture based on some computational experiments on the expected likelihood of such events.
Contingency tables are useful objects in statistics for representing 2-way data. With fixed row and column sums, and a total of $n$ entries, contingency tables correspond to parabolic double cosets of $S_n$. The uniform distribution on $S_n$ induces the Fisher-Yates distribution, classical for its use in the chi-squared test for independence. A Markov chain on $S_n$ can then induce a random process on the space of contingency tables through the double cosets correspondence. The random transpositions Markov chain on $S_n$ induces a natural `swap' Markov chain on the space of contingency tables;…
We present a new lower bound on the number of contingency tables, improving upon and extending previous lower bounds by Barvinok and Gurvits. As an application, we obtain new lower bounds on the volumes of flow and transportation polytopes. Our proofs are based on recent results on Lorentzian polynomials.
The Diaconis--Gangolli random walk is an algorithm that generates an almost uniform random graph with prescribed degrees. In this paper, we study the mixing time of the Diaconis--Gangolli random walk restricted on $n\times n$ contingency tables over $\mathbb{Z}/q\mathbb{Z}$. We prove that the random walk exhibits cutoff at $\frac{n^2}{4(1- \cos{\frac{2 π}{q}})} \log n, $ when $\log q=o\left (\frac{\sqrt{\log n}}{\log \log n}\right )$.
We obtain sharp asymptotic estimates on the number of $n \times n$ contingency tables with two linear margins $Cn$ and $BCn$. The results imply a second order phase transition on the number of such contingency tables, with a critical value at \ts $B_{c}:=1 + \sqrt{1+1/C}$. As a consequence, for \ts $B>B_{c}$, we prove that the classical \emph{independence heuristic} leads to a large undercounting.
A binary contingency table is an m x n array of binary entries with prescribed row sums r=(r_1,...,r_m) and column sums c=(c_1,...,c_n). The configuration model for uniformly sampling binary contingency tables proceeds as follows. First, label N=\sum_{i=1}^{m} r_i tokens of type 1, arrange them in m cells, and let the i-th cell contain r_i tokens. Next, label another set of tokens of type 2 containing N=\sum_{j=1}^{n}c_j elements arranged in n cells, and let the j-th cell contain c_j tokens. Finally, pair the type-1 tokens with the type-2 tokens by generating a random permutation until the tot…
We present a randomized approximation algorithm for counting contingency tables, mxn non-negative integer matrices with given row sums R=(r_1, ..., r_m) and column sums C=(c_1, ..., c_n). We define smooth margins (R,C) in terms of the typical table and prove that for such margins the algorithm has quasi-polynomial N^{O(ln N)} complexity, where N=r_1+...+r_m=c_1+...+c_n. Various classes of margins are smooth, e.g., when m=O(n), n=O(m) and the ratios between the largest and the smallest row sums as well as between the largest and the smallest column sums are strictly smaller than the golden rati…
Let R=(r_1, ..., r_m) and C=(c_1, ..., c_n) be positive integer vectors such that r_1 +... + r_m=c_1 +... + c_n. We consider the set Sigma(R, C) of non-negative mxn integer matrices (contingency tables) with row sums R and column sums C as a finite probability space with the uniform measure. We prove that a random table D in Sigma(R,C) is close with high probability to a particular matrix ("typical table'') Z defined as follows. We let g(x)=(x+1) ln(x+1)-x ln x for non-negative x and let g(X)=sum_ij g(x_ij) for a non-negative matrix X=(x_ij). Then g(X) is strictly concave and attains its maxim…
We give an algorithm that generates a uniformly random contingency table with specified marginals, i.e. a matrix with non-negative integer values and specified row and column sums. Such algorithms are useful in statistics and combinatorics. When $Δ^4< M/5$, where $Δ$ is the maximum of the row and column sums and $M$ is the sum of all entries of the matrix, our algorithm runs in time linear in $M$ in expectation. Most previously published algorithms for this problem are approximate samplers based on Markov chain Monte Carlo, whose provable bounds on the mixing time are typically polynomials wit…
We consider nonnegative integer matrices with specified row and column sums and upper bounds on the entries. We show that the logarithm of the number of such matrices is approximated by a concave function of the row and column sums. We give efficiently computable estimators for this function, including one suggested by a maximum-entropy random model; we show that these estimators are asymptotically exact as the dimension of the matrices goes to infinity. We finish by showing that, for kappa >= 2 and for sufficiently small row and column sums, the number of matrices with these row and column su…
Importance sampling has been reported to produce algorithms with excellent empirical performance in counting problems. However, the theoretical support for its efficiency in these applications has been very limited. In this paper, we propose a methodology that can be used to design efficient importance sampling algorithms for counting and test their efficiency rigorously. We apply our techniques after transforming the problem into a rare-event simulation problem--thereby connecting complexity analysis of counting problems with efficiency in the context of rare-event simulation. As an illustrat…
Given m positive integers R=(r_i), n positive integers C=(c_j) such that sum r_i = sum c_j =N, and mn non-negative weights W=(w_{ij}), we consider the total weight T=T(R, C; W) of non-negative integer matrices (contingency tables) D=(d_{ij}) with the row sums r_i, column sums c_j, and the weight of D equal to prod w_{ij}^{d_{ij}}. We present a randomized algorithm of a polynomial in N complexity which computes a number T'=T'(R,C; W) such that T' < T < alpha(R, C) T' where alpha(R,C) = min{prod r_i! r_i^{-r_i}, prod c_j! c_j^{-c_j}} N^N/N!. In many cases, ln T' provides an asymptotically accura…
For parameters $n,δ,B,$ and $C$, let $X=(X_{k\ell})$ be the random uniform contingency table whose first $\lfloor n^δ \rfloor $ rows and columns have margin $\lfloor BCn \rfloor$ and the last $n$ rows and columns have margin $\lfloor Cn \rfloor$. For every $0<δ<1$, we establish a sharp phase transition of the limiting distribution of each entry of $X$ at the critical value $B_{c}=1+\sqrt{1+1/C}$. In particular, for $1/2<δ<1$, we show that the distribution of each entry converges to a geometric distribution in total variation distance, whose mean depends sensitively on whether $B<B_{c}$ or $B>B…
We count mxn non-negative integer matrices (contingency tables) with prescribed row and column sums (margins). For a wide class of smooth margins we establish a computationally efficient asymptotic formula approximating the number of matrices within a relative error which approaches 0 as m and n grow.
