首页/论文/1908.10572

arXiv 1908.105722019-08-28stat.MEmath.STstat.MLstat.TH

On the overestimation of widely applicable Bayesian information criterion

Toru Imai

备注：: 12 pages

英文摘要：

A widely applicable Bayesian information criterion (Watanabe, 2013) is applicable for both regular and singular models in the model selection problem. This criterion tends to overestimate the log marginal likelihood. We identify an overestimating term of a widely applicable Bayesian information criterion. Adjustment of the term gives an asymptotically unbiased estimator of the leading two terms of asymptotic expansion of the log marginal likelihood. In numerical experiments on regular and singular models, the adjustment resulted in smaller bias than the original criterion.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Investigation of the widely applicable Bayesian information criterion

The widely applicable Bayesian information criterion (WBIC) is a simple and fast approximation to the model evidence that has received little practical consideration. WBIC uses the fact that the log evidence can be written as an expectation, with respect to a powered posterior proportional to the likelihood raised to a power $t^*\in{(0,1)}$, of the log deviance. Finding this temperature value $t^*$ is generally an intractable problem. We find that for a particular tractable statistical model that the mean squared error of an optimally-tuned version of WBIC with correct temperature $t^*$ is low…

2016-04-28stat.ME

Posterior Covariance Information Criterion for Weighted Inference

For predictive evaluation based on quasi-posterior distributions, we develop a new information criterion, the posterior covariance information criterion (PCIC. PCIC generalises the widely applicable information criterion WAIC so as to effectively handle predictive scenarios where likelihoods for the estimation and the evaluation of the model may be different. A typical example of such scenarios is the weighted likelihood inference, including prediction under covariate shift and counterfactual prediction. The proposed criterion utilises a posterior covariance form and is computed by using only …

2023-01-25math.STstat.MEstat.ML

A Widely Applicable Bayesian Information Criterion

Tokyo Institute of Technology（东京工业大学）

A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus logarithm of Bayes marginal likelihood, can be asymptotically approximated by the Schwarz Bayes information criterion (BIC), whereas in singular models such approximation does not hold. Recently, it was proved that the Bayes free energy of a singular model is asymptotically give…

2012-08-31cs.LGstat.ML

奇异条件下的学习：一种改进WBIC与sBIC的信息准则

Learning under Singularity: An Information Criterion improving WBIC and sBIC

Osaka University（大阪大学）

本文提出奇异条件下的学习（LS）信息准则，融合WAIC经验损失与sBIC式惩罚项，在无正则性约束下改进WBIC和sBIC的模型选择性能。

2024-02-22cs.LGstat.ML

连接WAIC和WBIC的渐近方程在奇异模型中

An Asymptotic Equation Linking WAIC and WBIC in Singular Models

TOYOTA CENTRAL R&D LABS., INC.（丰田中央研发实验室有限公司）；AISIN CORPORATION（日产公司）

本文推导了连接WAIC和WBIC的渐近方程，揭示了奇异模型中这两种标准的结构关系及渐近行为。

2025-11-25cs.LGmath.STstat.ML

多维高斯过程模型选择准则：应用于径向速度

A Model Selection Criterion for Multidimensional Gaussian Processes: Application to Radial Velocities

University of Warwick（华威大学）；University of Oxford（牛津大学）

针对多维高斯过程回归中联合建模辅助活动指标以分离恒星和行星信号的问题，提出一种基于条件径向速度似然和有效参数计数的信息准则MGIC_rv，用于比较不同多GP模型并识别最有效约束RV信号的活动指标。

2026-06-08astro-ph.EPastro-ph.IMastro-ph.SR

更多相关论文收起更多论文

适用于一般损失函数的后验协方差信息准则

Posterior covariance information criterion for general loss functions

本文提出利用后验协方差以低成本估计广义贝叶斯推断的一般预测度量，给出 Gibbs 与插件泛化误差估计量、理论保证及多类应用。

2025-01-11stat.ME

Rebuilding Factorized Information Criterion: Asymptotically Accurate Marginal Likelihood

National Institute of Informatics（国立信息学研究所）；Kyoto University（京都大学）；NEC Knowledge Discovery Laboratories（NEC知识发现实验室）

Factorized information criterion (FIC) is a recently developed approximation technique for the marginal log-likelihood, which provides an automatic model selection framework for a few latent variable models (LVMs) with tractable inference algorithms. This paper reconsiders FIC and fills theoretical gaps of previous FIC studies. First, we reveal the core idea of FIC that allows generalization for a broader class of LVMs, including continuous LVMs, in contrast to previous FICs, which are applicable only to binary LVMs. Second, we investigate the model selection mechanism of the generalized FIC. …

2015-04-22cs.LGstat.ML

关于某些贝叶斯模型评估统计量的大样本极限

On the large-sample limits of some Bayesian model evaluation statistics

本文研究贝叶斯模型评估准则（DIC、BPIC、WBIC）的大样本几乎必然极限，建立后验与广义后验一致性的新结果，并通过实例验证其有效性。

2025-10-28math.STstat.TH

通过贝叶斯广义融合lasso开发空间数据分析的信息准则

Developing an information criterion for spatial data analysis through Bayesian generalized fused lasso

针对空间变化系数模型中贝叶斯广义融合lasso先验分布复杂度确定难题，推导估计量渐近性质并提出先验增广信息准则PIIC，数值实验证明其在降低预测风险上优于WAIC。

2025-10-13stat.ME

二维奇异模型学习系数的精确代数计算

Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models

Sorbonne Université（索邦大学）；Inria（法国国家信息与自动化研究院）

本文针对二维奇异模型，提出首个可精确计算局部实对数典范阈值（RLCT）的确定性算法，为校准采样估计器提供真值，在浅层区域比采样方法更快。

2026-08-21cs.LGcs.SCmath.AG

Signal Denoising Using the Minimum-Probability-of-Error Criterion

We address the problem of signal denoising via transform-domain shrinkage based on a novel $\textit{risk}$ criterion called the minimum probability of error (MPE), which measures the probability that the estimated parameter lies outside an $ε$-neighborhood of the actual value. However, the MPE, similar to the mean-squared error (MSE), depends on the ground-truth parameter, and has to be estimated from the noisy observations. We consider linear shrinkage-based denoising functions, wherein the optimum shrinkage parameter is obtained by minimizing an estimate of the MPE. When the probability of e…

2017-02-25stat.AP

面向贝叶斯序列数据模型的协方差校正WAIC

Covariance-Corrected WAIC for Bayesian Sequential Data Models

本文提出协方差校正WAIC（CC-WAIC），通过纳入对数似然后验协方差结构改进贝叶斯序列数据模型选择，并引入基于有效样本量的偏差校正，经模拟实验验证其在不同时间依赖水平下的可靠性。

2025-09-22stat.COstat.ME

使用经验损失估计学习系数

Estimation of the Learning Coefficient Using Empirical Loss

针对学习系数通常难以闭式求解的问题，本文提出基于 Watanabe 所引入 Empirical Loss 的新型数值估计法，并通过理论与实验证明其相较 Watanabe 和 Imai 方法具有更低偏差与方差。

2025-02-14cs.LGstat.ML

信息论安全贝叶斯优化

Information-Theoretic Safe Bayesian Optimization

Bosch Corporate Research（博世公司研究中心）；TU Darmstadt（达姆施塔特工业大学）；German Research Center for AI (DFKI)（德国人工智能研究中心）

针对连续域中带未知安全约束的贝叶斯优化问题，提出直接利用 GP 后验的信息论安全探索准则，无需额外超参数，并在理论上保证高概率安全与任意精度收敛，实验显示数据效率和可扩展性提升。

2024-05-10cs.AIcs.LGstat.ML

评估贝叶斯诊断分类模型选择中WAIC与PSIS-LOO的表现

Evaluating WAIC and PSIS-LOO for Bayesian Diagnostic Classification Model Selection

本研究通过模拟比较WAIC、PSIS-LOO与DIC在贝叶斯诊断分类模型选择中的表现，发现前两者通常更倾向真实模型但偶有不一致，并给出按模型复杂度与先验信息选择指标的建议。

2024-10-03stat.AP

文化数据中显著主成分数量的贝叶斯估计

Bayesian estimation of the number of significant principal components for cultural data

提出基于变分贝叶斯PCA的新方法，用于在样本数接近或大于特征数时估计显著主成分数量，避免手动猜测和方差高估，适用于多种数据类型，测试表明不会高估主成分数，广泛适用于生命科学。

2024-09-18stat.AP

Prior Intensified Information Criterion

The widely applicable information criterion (WAIC) has been used as a model selection criterion for Bayesian statistics in recent years. It is an asymptotically unbiased estimator of the Kullback-Leibler divergence between a Bayesian predictive distribution and the true distribution. Not only is the WAIC theoretically more sound than other information criteria, its usefulness in practice has also been reported. On the other hand, the WAIC is intended for settings in which the prior distribution does not have an asymptotic influence, and as we set the class of the prior distribution to be more …

2022-08-07stat.ME