A test for equality of multinomial distributions vs increasing convex order

Two-Sample Test for Sparse High Dimensional Multinomial Distributions

In this paper we consider testing the equality of probability vectors of two independent multinomial distributions in high dimension. The classical chi-square test may have some drawbacks in this case since many of cell counts may be zero or may not be large enough. We propose a new test and show its asymptotic normality and the asymptotic power function. Based on the asymptotic power function, we present an application of our result to neighborhood type test which has been previously studied, especially for the case of fairly small $p$-values. To compare the proposed test with existing tests,…

Pearson卡方统计量的非渐近近似及其在离散分布概率权重严格凸函数置信区间中的应用

Non-asymptotic approximations for Pearson's chi-square statistic and its application to confidence intervals for strictly convex functions of the probability weights of discrete distributions

本文提出非渐近局部正态近似，用于Pearson卡方统计量，并推导全变差界及分位数耦合不等式，进而为离散分布的负熵等严格凸函数构建置信区间。

A Generalization of the Savage-Dickey Density Ratio for Testing Equality and Order Constrained Hypotheses

The Savage-Dickey density ratio is a specific expression of the Bayes factor when testing a precise (equality constrained) hypothesis against an unrestricted alternative. The expression greatly simplifies the computation of the Bayes factor at the cost of assuming a specific form of the prior under the precise hypothesis as a function of the unrestricted prior. A generalization was proposed by Verdinelli and Wasserman (1995) such that the priors can be freely specified under both hypotheses while keeping the computational advantage. This paper presents an extension of this generalization when …

Multinomial and empirical likelihood under convex constraints: directions of recession, Fenchel duality, perturbations

The primal problem of multinomial likelihood maximization restricted to a convex closed subset of the probability simplex is studied. Contrary to widely held belief, a solution of this problem may assign a positive mass to an outcome with zero count. Related flaws in the simplified Lagrange and Fenchel dual problems, which arise because the recession directions are ignored, are identified and corrected. A solution of the primal problem can be obtained by the PP (perturbed primal) algorithm, that is, as the limit of a sequence of solutions of perturbed primal problems. The PP algorithm may be i…

Adjusted chi-square test for degree-corrected block models

We propose a goodness-of-fit test for degree-corrected stochastic block models (DCSBM). The test is based on an adjusted chi-square statistic for measuring equality of means among groups of $n$ multinomial distributions with $d_1,\dots,d_n$ observations. In the context of network models, the number of multinomials, $n$, grows much faster than the number of observations, $d_i$, corresponding to the degree of node $i$, hence the setting deviates from classical asymptotics. We show that a simple adjustment allows the statistic to converge in distribution, under null, as long as the harmonic mean …

高维多项分布的检验及其在文本分析中的应用

Testing High-dimensional Multinomials with Applications to Text Analysis

本文研究$K$组高维多项分布概率质量函数相等的检验问题，提出一种具有渐近标准正态分布的检验统计量并达到最优检测边界，模拟与真实数据集验证了其有效性。