用于贝叶斯模型批评的留存预测检验
Holdout predictive checks for Bayesian model criticism
AI总结：
针对后验预测检验因重复使用数据导致p值未校准的问题，提出结合贝叶斯建模与频率学派评估的留存预测检验方法，证明其校准性并在多场景验证。
摘要：
贝叶斯建模有助于应用研究人员明确阐述关于其数据的假设，并开发针对特定应用定制的模型。得益于成熟的近似后验推断方法，研究人员如今能够轻松构建、使用和修正适用于大规模丰富数据的复杂贝叶斯模型。然而，这些能力也让模型批评问题凸显出来。研究人员需要工具来诊断模型的适配性，了解模型的不足之处，并指导模型修正。本文提出了一种新的贝叶斯模型批评方法：留存预测检验（holdout predictive check, HPC）。HPC 建立在后验预测检验（posterior predictive checks, PPCs）的基础上，后者是一种通过评估观测数据上的后验预测分布来检验模型的经典方法。但 PPC 会两次使用数据——既用于计算后验预测分布，又用于评估该分布——这可能导致 $p$ 值未校准。相比之下，HPC 将后验预测分布与来自总体分布的样本（即留存数据集）进行比较。该方法将贝叶斯建模与频率学派评估相结合。我们证明，与 PPC 不同，HPC 具有恰当的校准性。在实证研究中，我们在经典回归、文本数据层次模型以及因子分析场景下对 HPC 进行了研究。
英文摘要：
Bayesian modeling helps applied researchers articulate assumptions about their data and develop models tailored for specific applications. Thanks to good methods for approximate posterior inference, researchers can now easily build, use, and revise complicated Bayesian models for large and rich data. These capabilities, however, bring into focus the problem of model criticism. Researchers need tools to diagnose the fitness of their models, to understand where they fall short, and to guide their revision. In this paper we develop a new method for Bayesian model criticism, the holdout predictive check (HPC). HPCs are built on posterior predictive checks (PPCs), a seminal method that checks a model by assessing the posterior predictive distribution on the observed data. However, PPCs use the data twice---both to calculate the posterior predictive and to evaluate it---which can lead to uncalibrated $p$-values. HPCs, in contrast, compare the posterior predictive distribution to a draw from the population distribution, a heldout dataset. This method blends Bayesian modeling with frequentist assessment. Unlike the PPC, we prove that the HPC is properly calibrated. Empirically, we study HPCs on classical regression, a hierarchical model of text data, and factor analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
