We propose and analyse a novel nonparametric goodness of fit testing procedure for exchangeable exponential random graph models (ERGMs) when a single network realisation is observed. The test determines how likely it is that the observation is generated from a target unnormalised ERGM density. Our test statistics are derived from a kernel Stein discrepancy, a divergence constructed via Steins method using functions in a reproducing kernel Hilbert space, combined with a discrete Stein operator for ERGMs. The test is a Monte Carlo test based on simulated networks from the target ERGM. We show theoretical properties for the testing procedure for a class of ERGMs. Simulation studies and real network applications are presented.
Non-parametric goodness-of-fit testing procedures based on kernel Stein discrepancies (KSD) are promising approaches to validate general unnormalised distributions in various scenarios. Existing works focused on studying kernel choices to boost test performances. However, the choices of (non-unique) Stein operators also have considerable effect on the test performances. Inspired by the standardisation technique that was originally developed to better derive approximation properties for normal distributions, we present a unifying framework, called standardisation-function kernel Stein discrepan…
We derive a new discrepancy statistic for measuring differences between two probability distributions based on combining Stein's identity with the reproducing kernel Hilbert space theory. We apply our result to test how well a probabilistic model fits a set of observations, and derive a new class of powerful goodness-of-fit tests that are widely applicable for complex and high dimensional distributions, even for those with computationally intractable normalization constants. Both theoretical and empirical properties of our methods are studied thoroughly.
In many applications, we encounter data on Riemannian manifolds such as torus and rotation groups. Standard statistical procedures for multivariate data are not applicable to such data. In this study, we develop goodness-of-fit testing and interpretable model criticism methods for general distributions on Riemannian manifolds, including those with an intractable normalization constant. The proposed methods are based on extensions of kernel Stein discrepancy, which are derived from Stein operators on Riemannian manifolds. We discuss the connections between the proposed tests with existing ones …
In many fields, data appears in the form of direction (unit vector) and usual statistical procedures are not applicable to such directional data. In this study, we propose non-parametric goodness-of-fit testing procedures for general directional distributions based on kernel Stein discrepancy. Our method is based on Stein's operator on spheres, which is derived by using Stokes' theorem. Notably, the proposed method is applicable to distributions with an intractable normalization constant, which commonly appear in directional statistics. Experimental results demonstrate that the proposed method…
Department of Biostatistics, Harvard（哈佛大学生物统计学系）；Microsoft Research New England（微软新英格兰研究院）
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational cost that grows quadratically in the sample size. While linear-time Stein discrepancies have been proposed for goodness-of-fit testing, they exhibit avoidable degradations in testing power -- even when power is explicitly optimized. To address these shortcomings, we introduce featu…
University of Cambridge（剑桥大学）；Microsoft Research Cambridge（微软研究院剑桥分部）；Imperial College London（帝国理工学院）；The Alan Turing Institute（阿兰·图灵研究所）
Kernelized Stein discrepancy (KSD), though being extensively used in goodness-of-fit tests and model learning, suffers from the curse-of-dimensionality. We address this issue by proposing the sliced Stein discrepancy and its scalable and kernelized variants, which employ kernel-based test functions defined on the optimal one-dimensional projections. When applied to goodness-of-fit tests, extensive experiments show the proposed discrepancy significantly outperforms KSD and various baselines in high dimensions. For model learning, we show its advantages over existing Stein discrepancy baselines …
University of Cambridge（剑桥大学）；Imperial College London（帝国理工学院）
Sliced Stein discrepancy (SSD) and its kernelized variants have demonstrated promising successes in goodness-of-fit tests and model learning in high dimensions. Despite their theoretical elegance, their empirical performance depends crucially on the search of optimal slicing directions to discriminate between two distributions. Unfortunately, previous gradient-based optimisation approaches for this task return sub-optimal results: they are computationally expensive, sensitive to initialization, and they lack theoretical guarantees for convergence. We address these issues in two steps. First, w…
We propose a class of weighted $L_2$-type tests of fit to the Gamma distribution. Our novel procedure is based on a fixed point property of a new transformation connected to a Steinian characterization of the family of Gamma distributions. We derive the weak limits of the statistic under the null hypothesis and under contiguous alternatives. Further, we establish the global consistency of the tests and apply a parametric bootstrap technique in a Monte Carlo simulation study to show the competitiveness to existing procedures.
Novel Stein-type Characterizations of Bivariate Count Distributions with Applications
School of Mathematics, South China University of Technology（华南理工大学数学学院）；Department of Mathematics and Statistics, Helmut Schmidt University（赫尔穆特·施密特大学数学与统计系）
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepancies (KSDs). While kernels make Stein discrepancies tractable, they pose several challenges in high dimensions. We introduce kernelized complete conditional Stein discrepancies (KCC-SDs). Complete conditionals turn a multivariate distribution into multiple univariate distributions…
Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling. The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control. However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical. This paper focuses on posterior approximation using Stein …
University of Oxford（牛津大学）；The Alan Turing Institute（阿兰·图灵研究所）
Score-based kernelised Stein discrepancy (KSD) tests have emerged as a powerful tool for the goodness of fit tests, especially in high dimensions; however, the test performance may depend on the choice of kernels in an underlying reproducing kernel Hilbert space (RKHS). Here we assess the effect of RKHS choice for KSD tests of random networks models, developed for exponential random graph models (ERGMs) in Xu and Reinert (2021)and for synthetic graph generators in Xu and Reinert (2022). We investigate the power performance and the computational runtime of the test in different scenarios, inclu…
Gatsby Unit, UCL（伦敦大学学院盖茨比单元）；CMAP, École Polytechnique（巴黎综合理工学院应用数学中心）；The Institute of Statistical Mathematics（统计数理研究所）
We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples. We learn the test features that best indicate the differences between observed samples and a reference model, by minimizing the false negative rate. These features are constructed via Stein's method, meaning that it is not necessary to compute the normalising constant of the model. We analyse the asymptotic Bahadur efficiency of the new test, and prove that under a mean-shift alternative, our test always has greater relative efficiency than a previous linear-time kernel test, regardles…
University of Toronto（多伦多大学）；Vector Institute（向量研究院）
We present a new method for evaluating and training unnormalized density models. Our approach only requires access to the gradient of the unnormalized model's log-density. We estimate the Stein discrepancy between the data density $p(x)$ and the model density $q(x)$ defined by a vector function of the data. We parameterize this function with a neural network and fit its parameters to maximize the discrepancy. This yields a novel goodness-of-fit test which outperforms existing methods on high dimensional data. Furthermore, optimizing $q(x)$ to minimize this discrepancy produces a novel method f…
Queensland University of Technology（昆士兰科技大学）；Centre for Data Science, Queensland University of Technology（昆士兰科技大学数据科学中心）；Quantium；ARC Centre of Excellence for the Mathematical Analysis of Cellular Systems (MACSYS)（ARC细胞系统数学分析卓越中心）
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We show that two classes of Maximum Mean Discrepancy (MMD) based tests attain this optimality on $\mathbb R^d$, while the quadratic-time Kernel Stein Discrepancy (KSD) based tests achieve the maximum exponential decay rate under a relaxed level constraint. Under the same performance…
Copula-Stein Discrepancy: A Generator-Based Stein Operator for Archimedean Dependence
Department of Mathematics, University of Louisiana at Lafayette, Lafayette, Louisiana, USA（路易斯安那州拉法叶大学数学系）；Department of Statistics, formerly of Presidency College, Kolkata, India（加德满都大学统计系）
Google Research（谷歌研究院）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；University College London（伦敦大学学院）
We propose two nonparametric statistical tests of goodness of fit for conditional distributions: given a conditional probability density function $p(y|x)$ and a joint sample, decide whether the sample is drawn from $p(y|x)r_x(x)$ for some density $r_x$. Our tests, formulated with a Stein operator, can be applied to any differentiable conditional density model, and require no knowledge of the normalizing constant. We show that 1) our tests are consistent against any fixed alternative conditional model; 2) the statistics can be estimated easily, requiring no density estimation as an intermediate…
Controlling Moments with Kernel Stein Discrepancies
Newcastle University（纽卡斯尔大学）；University College London（伦敦大学学院）；Gatsby Computational Neuroscience Unit, UCL（伦敦大学学院盖茨比计算神经科学单元）；Microsoft Research New England（微软研究院新英格兰分部）
Centre for Artificial Intelligence, University College London（大学学院伦敦大学人工智能中心）；Gatsby Computational Neuroscience Unit, University College London（大学学院伦敦大学盖茨比计算神经科学单位）；Department of Statistical Science, University College London（大学学院伦敦大学统计科学系）；Faculty of Engineering and Science, Adolfo Ibañez University（工程与科学学院，阿道弗·伊巴涅斯大学）
Synthetic data generation has become a key ingredient for training machine learning procedures, addressing tasks such as data augmentation, analysing privacy-sensitive data, or visualising representative samples. Assessing the quality of such synthetic data generators hence has to be addressed. As (deep) generative models for synthetic data often do not admit explicit probability distributions, classical statistical procedures for assessing model goodness-of-fit may not be applicable. In this paper, we propose a principled procedure to assess the quality of a synthetic data generator. The proc…
Centre for Artificial Intelligence（人工智能中心）；Gatsby Computational Neuroscience Unit（盖茨比计算神经科学研究组）；University College London（伦敦大学学院）；Inria London（法国国家信息与自动化研究所伦敦分所）
This paper studies goodness of fit tests and specification tests for an extension of the log-GARCH model which is stable by scaling. A Lagrange-Multiplier test is derived for testing the null assumption of extended log-GARCH against more general formulations including the Exponential GARCH (EGARCH). The null assumption of an EGARCH is also tested. Portmanteau goodness-of-fit tests are developed for the extended log-GARCH. Simulations illustrating the theoretical results and an application to real financial data are proposed.
University College London（伦敦大学学院）；University of Cambridge（剑桥大学）
Survival Analysis and Reliability Theory are concerned with the analysis of time-to-event data, in which observations correspond to waiting times until an event of interest such as death from a particular disease or failure of a component in a mechanical system. This type of data is unique due to the presence of censoring, a type of missing data that occurs when we do not observe the actual time of the event of interest but, instead, we have access to an approximation for it given by random interval in which the observation is known to belong. Most traditional methods are not designed to deal …
The Polynomial Stein Discrepancy for Assessing Moment Convergence
School of Mathematical Sciences, Queensland University of Technology, Australia（昆士兰技术大学数学科学学院，澳大利亚）；Centre for Data Science, Queensland University of Technology, Australia（昆士兰技术大学数据科学中心，澳大利亚）；School of Mathematics and Physics, University of Queensland, Brisbane, Australia（昆士兰大学数学与物理学院，布里斯班，澳大利亚）
We study the conditional distribution of goodness of fit statistics of the Cramér--von Mises type given the complete sufficient statistics in testing for exponential family models. We show that this distribution is close, in large samples, to that given by parametric bootstrapping, namely, the unconditional distribution of the statistic under the value of the parameter given by the maximum likelihood estimate. As part of the proof, we give uniform Edgeworth expansions of Rao--Blackwell estimates in these models.
We introduce tests for the goodness of fit of point patterns via methods from topological data analysis. More precisely, the persistent Betti numbers give rise to a bivariate functional summary statistic for observed point patterns that is asymptotically Gaussian in large observation windows. We analyze the power of tests derived from this statistic on simulated point patterns and compare its performance with global envelope tests. Finally, we apply the tests to a point pattern from an application context in neuroscience. As the main methodological contribution, we derive sufficient conditions…
