We consider the problem of hypothesis testing in the situation when the first hypothesis is simple and the second one is local one-sided composite. We describe the choice of the thresholds and the power functions of the Score Function test, of the General Likelihood Ratio test, of the Wald test and of two Bayes tests in the situation when the intensity function of the observed inhomogeneous Poisson process is smooth with respect to the parameter. It is shown that almost all these tests are asymptotically uniformly most powerful. The results of numerical simulations are presented.
We consider the problem of hypothesis testing in the situation where the first hypothesis is simple and the second one is local one-sided composite. We describe the choice of the thresholds and the power functions of different tests when the intensity function of the observed inhomogeneous Poisson process has two different types of singularity: cusp and discontinuity. The asymptotic results are illustrated by numerical simulations.
Methods based on hypothesis tests (HTs) in the Haar domain are widely used to denoise Poisson count data. Facing large datasets or real-time applications, Haar-based denoisers have to use the decimated transform to meet limited-memory or computation-time constraints. Unfortunately, for regular underlying intensities, decimation yields discontinuous estimates and strong "staircase" artifacts. In this paper, we propose to combine the HT framework with the decimated biorthogonal Haar (Bi-Haar) transform instead of the classical Haar. The Bi-Haar filter bank is normalized such that the p-values of…
This paper focuses on hypothesis testing for the input of a Lévy-driven storage system by sampling of the storage level. As the likelihood is not explicit we propose two tests that rely on transformation of the data. The first approach uses i.i.d. `quasi-busy-periods' between observations of zero workload. The distribution of the duration of quasi-busy-periods is determined. The second method is a conditional likelihood ratio test based on the Bernoulli events of observing a zero or positive workload, conditional on the previous workload. Performance analysis is presented for both tests along …
We consider the problems of hypothesis testing on a probability measure of independent sample, on solution of ill-posed problem, on deconvolution problem and on Poisson mean measure. For all these setups necessary conditions and sufficient conditions are given for distinguishability of sets of hypothesis. In the case of hypothesis testing on a probability measure and on Poisson mean measure the results are given in terms of weak topology and topology of weak convergence on all Borel sets. The problem of discernibility of hypothesis is also studied. In other cases the necessary and sufficient c…
We propose the goodness of fit test for inhomogeneous Poisson processes with unknown scale and shift parameters. A test statistic of Cramer-von Mises type is proposed and its asymptotic behavior is studied. We show that under null hypothesis the limit distribution of this statistic does not depend on unknown parameters.
We study exact confidence intervals and two-sided hypothesis tests for univariate parameters of stochastically increasing discrete distributions, such as the binomial and Poisson distributions. It is shown that several popular methods for constructing short intervals lack strict nestedness, meaning that accepting a lower confidence level not always will lead to a shorter confidence interval. These intervals correspond to a class of tests that are shown to assign differing $p$-values to indistinguishable models. Finally, we show that among strictly nested intervals, fiducial intervals, includin…
The University of California, San Francisco（加州大学旧金山分校）；Tongji University（同济大学）；Syracuse University（雪城大学）
We study estimation and testing in the Poisson regression model with noisy high dimensional covariates, which has wide applications in analyzing noisy big data. Correcting for the estimation bias due to the covariate noise leads to a non-convex target function to minimize. Treating the high dimensional issue further leads us to augment an amenable penalty term to the target function. We propose to estimate the regression parameter through minimizing the penalized target function. We derive the L1 and L2 convergence rates of the estimator and prove the variable selection consistency. We further…
We consider the problem of parameter estimation by observations of inhomogeneous Poisson process. It is well-known that if the regularity conditions are fulfilled then the maximum likelihood and Bayesian estimators are consistent, asymptotically normal and asymptotically efficient. These regularity conditions can be roughly presented as follows: a) the intensity function of observed process belongs to known parametric family of functions, b) the model is identifiable, c) the Fisher information is positive continuous function, d) the intensity function is sufficiently smooth with respect to the…
Large sample size equivalence between the celebrated {\it approximated} Good-Turing estimator of the probability to discover a species already observed a certain number of times (Good, 1953) and the modern Bayesian nonparametric counterpart has been recently established by virtue of a particular smoothing rule based on the two-parameter Poisson-Dirichlet model. Here we improve on this result showing that, for any finite sample size, when the population frequencies are assumed to be selected from a superpopulation with two-parameter Poisson-Dirichlet distribution, then Bayesian nonparametric es…
In this paper we consider Poisson loglinear models with linear constraints (LMLC) on the expected table counts. Multinomial and product multinomial loglinear models can be obtained by considering that some marginal totals (linear constraints on the expected table counts) have been prefixed in a Poisson loglinear model. Therefore with the theory developed in this paper, multinomial and product multinomial loglinear models can be considered as a particular case. To carry out inferences on the parameters in the LMLC an information-theoretic approach is followed from which the classical maximum li…
We consider together the retrospective and the sequential change-point detection in a general class of integer-valued time series. The conditional mean of the process depends on a parameter $θ^*$ which may change over time. We propose procedures which are based on the Poisson quasi-maximum likelihood estimator of the parameter, and where the updated estimator is computed without the historical observations in the sequential framework. For both the retrospective and the sequential detection, the test statistics converge to some distributions obtained from the standard Brownian motion under the …
We describe the utility of point processes and failure rates and the most common point process for modeling failure rates, the Poisson point process. Next, we describe the uniformly most powerful test for comparing the rates of two Poisson point processes for a one-sided test (henceforth referred to as the "rate test"). A common argument against using this test is that real world data rarely follows the Poisson point process. We thus investigate what happens when the distributional assumptions of tests like these are violated and the test still applied. We find a non-pathological example (usin…
Bayesian supervised predictive classifiers, hypothesis testing, and parametric estimation under Partition Exchangeability are implemented. The two classifiers presented are the marginal classifier (that assumes test data is i.i.d.) next to a more computationally costly but accurate simultaneous classifier (that finds a labelling for the entire test dataset at once based on simultanous use of all the test data to predict each label). We also provide the Maximum Likelihood Estimation (MLE) of the only underlying parameter of the partition exchangeability generative model as well as hypothesis te…
We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity. The alternatives are stationary self-exciting point processes. We consider one-sided parametric and one-sided nonparametric composite alternatives and construct locally asymptotically uniformly most powerful tests. The results of numerical simulations of the tests are presented.
We consider a class of observation-driven Poisson count processes where the current value of the accompanying intensity process depends on previous values of both processes. We show under a contractive condition that the bivariate process has a unique stationary distribution and that a stationary version of the count process is absolutely regular. Moreover, since the intensities can be written as measurable functionals of the count variables, we conclude that the bivariate process is ergodic. As an important application of these results, we show how a test method previously used in the case of…
We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity against a composite one-sided parametric alternative that this is a self-correcting point process. The underlying family of measures is locally asymptotically quadratic and we describe the behavior of score function, likelihood ratio and Wald tests in the asymptotics of large samples. The results of numerical simulations are presented.
In this work, based on a realization of an inhomogeneous Poisson process whose intensity function depends on a real unknown parameter, we consider a simple hypothesis against a sequence of close (contiguous) alternatives. Under certain regularity conditions we obtain the power loss of the score test with respect to the Neyman-Pearson test. The power loss measures the performance of a second order efficient test by the help of third order asymptotic properties of the problem under consideration.
First-order separability of a spatio-temporal point process plays a fundamental role in the analysis of spatio-temporal point pattern data. While it is often a convenient assumption that simplifies the analysis greatly, existing non-separable structures should be accounted for in the model construction. We propose three different tests to investigate this hypothesis as a step of preliminary data analysis. The first two tests are exact or asymptotically exact for Poisson processes. The first test based on permutations and global envelopes allows us to detect at which spatial and temporal locati…
Long-range dependence induced by heavy tails is a widely reported feature of internet traffic. Long-range dependence can be defined as the regular variation of the variance of the integrated process, and half the index of regular variation is then referred to as the Hurst index. The infinite-source Poisson process (a particular case of which is the $M/G/\infty$ queue) is a simple and popular model with this property, when the tail of the service time distribution is regularly varying. The Hurst index of the infinite-source Poisson process is then related to the index of regular variation of th…
University of Edinburgh（爱丁堡大学）；University of California, San Diego（加州大学圣地亚哥分校）
We give an algorithm for properly learning Poisson binomial distributions. A Poisson binomial distribution (PBD) of order $n$ is the discrete probability distribution of the sum of $n$ mutually independent Bernoulli random variables. Given $\widetilde{O}(1/ε^2)$ samples from an unknown PBD $\mathbf{p}$, our algorithm runs in time $(1/ε)^{O(\log \log (1/ε))}$, and outputs a hypothesis PBD that is $ε$-close to $\mathbf{p}$ in total variation distance. The previously best known running time for properly learning PBDs was $(1/ε)^{O(\log(1/ε))}$. As one of our main contributions, we provide a novel…
Although much research has been devoted to the problem of restoring Poissonian images, namely in the fields of medical and astronomical imaging, applying the state of the art regularizers (such as those based on wavelets or total variation) to this class of images is still an open research front. This paper proposes a new image deconvolution approach for images with Poisson statistical models, with the following building blocks: (a) a standard regularization/MAP criterion, combining the Poisson log-likelihood with a regularizer (log-prior) is adopted; (b) the resulting optimization problem (wh…
