On Hypothesis Testing for Poisson Processes. Singular Cases

Exact confidence intervals and hypothesis tests for parameters of discrete distributions

We study exact confidence intervals and two-sided hypothesis tests for univariate parameters of stochastically increasing discrete distributions, such as the binomial and Poisson distributions. It is shown that several popular methods for constructing short intervals lack strict nestedness, meaning that accepting a lower confidence level not always will lead to a shorter confidence interval. These intervals correspond to a class of tests that are shown to assign differing $p$-values to indistinguishable models. Finally, we show that among strictly nested intervals, fiducial intervals, includin…

Hypothesis testing near singularities and boundaries

The likelihood ratio statistic, with its asymptotic $χ^2$ distribution at regular model points, is often used for hypothesis testing. At model singularities and boundaries, however, the asymptotic distribution may not be $χ^2$, as highlighted by recent work of Drton. Indeed, poor behavior of a $χ^2$ for testing near singularities and boundaries is apparent in simulations, and can lead to conservative or anti-conservative tests. Here we develop a new distribution designed for use in hypothesis testing near singularities and boundaries, which asymptotically agrees with that of the likelihood rat…

On High dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization

We study estimation and testing in the Poisson regression model with noisy high dimensional covariates, which has wide applications in analyzing noisy big data. Correcting for the estimation bias due to the covariate noise leads to a non-convex target function to minimize. Treating the high dimensional issue further leads us to augment an amenable penalty term to the target function. We propose to estimate the regression parameter through minimizing the penalized target function. We derive the L1 and L2 convergence rates of the estimator and prove the variable selection consistency. We further…

在奇异模型中可观测域上的假设检验

Hypothesis Testing over Observable Regimes in Singular Models

本文研究了奇异模型中假设检验的可行性，指出非识别参数的假设导致检验障碍，而基于可识别可观测值的假设则回归经典检验理论。

Breaking hypothesis testing for failure rates

We describe the utility of point processes and failure rates and the most common point process for modeling failure rates, the Poisson point process. Next, we describe the uniformly most powerful test for comparing the rates of two Poisson point processes for a one-sided test (henceforth referred to as the "rate test"). A common argument against using this test is that real world data rarely follows the Poisson point process. We thus investigate what happens when the distributional assumptions of tests like these are violated and the test still applied. We find a non-pathological example (usin…

Bayesian Generalization Error of Poisson Mixture and Simplex Vandermonde Matrix Type Singularity

A Poisson mixture is one of the practically important models in computer science, biology, and sociology. However, the theoretical property has not been studied because the posterior distribution can not be approximated by any normal distribution. Such a model is called singular and it is known that Real Log Canonical Threshold (RLCT) is equal to the coefficient of the asymptotically main term of the Bayesian generalization error. In this paper, we derive RLCT of a simplex Vandermonde matrix type singularity which is equal to that of a Poisson mixture in general cases.

Hypotheses Testing: Poisson Versus Self-exciting

We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity. The alternatives are stationary self-exciting point processes. We consider one-sided parametric and one-sided nonparametric composite alternatives and construct locally asymptotically uniformly most powerful tests. The results of numerical simulations of the tests are presented.

Hypotheses Testing: Poisson Versus Self-correcting

We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity against a composite one-sided parametric alternative that this is a self-correcting point process. The underlying family of measures is locally asymptotically quadratic and we describe the behavior of score function, likelihood ratio and Wald tests in the asymptotics of large samples. The results of numerical simulations are presented.

Power Loss for Inhomogeneous Poisson Processes

In this work, based on a realization of an inhomogeneous Poisson process whose intensity function depends on a real unknown parameter, we consider a simple hypothesis against a sequence of close (contiguous) alternatives. Under certain regularity conditions we obtain the power loss of the score test with respect to the Neyman-Pearson test. The power loss measures the performance of a second order efficient test by the help of third order asymptotic properties of the problem under consideration.

Testing the first-order separability hypothesis for spatio-temporal point patterns

First-order separability of a spatio-temporal point process plays a fundamental role in the analysis of spatio-temporal point pattern data. While it is often a convenient assumption that simplifies the analysis greatly, existing non-separable structures should be accounted for in the model construction. We propose three different tests to investigate this hypothesis as a step of preliminary data analysis. The first two tests are exact or asymptotically exact for Poisson processes. The first test based on permutations and global envelopes allows us to detect at which spatial and temporal locati…

Properly Learning Poisson Binomial Distributions in Almost Polynomial Time

We give an algorithm for properly learning Poisson binomial distributions. A Poisson binomial distribution (PBD) of order $n$ is the discrete probability distribution of the sum of $n$ mutually independent Bernoulli random variables. Given $\widetilde{O}(1/ε^2)$ samples from an unknown PBD $\mathbf{p}$, our algorithm runs in time $(1/ε)^{O(\log \log (1/ε))}$, and outputs a hypothesis PBD that is $ε$-close to $\mathbf{p}$ in total variation distance. The previously best known running time for properly learning PBDs was $(1/ε)^{O(\log(1/ε))}$. As one of our main contributions, we provide a novel…

Fast Algorithm for Low-rank matrix recovery in Poisson noise

This paper describes a fast algorithm for recovering low-rank matrices from their linear measurements contaminated with Poisson noise: the Poisson noise Maximum Likelihood Singular Value thresholding (PMLSV) algorithm. We propose a convex optimization formulation with a cost function consisting of the sum of a likelihood function and a regularization function which the nuclear norm of the matrix. Instead of solving the optimization problem directly by semi-definite program (SDP), we derive an iterative singular value thresholding algorithm by expanding the likelihood function. We demonstrate t…