arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.26733cs.LGstat.ML
收藏

神经后验估计的交叉拟合

Cross-Fitting for Neural Posterior Estimation

University of Michigan(密歇根大学)

Jeffrey Regier

AI总结:

针对标准神经后验估计(NPE)的过自信与权衡问题,提出交叉拟合NPE(CF-NPE),通过样本拆分与$K$折交叉拟合分离条件均值与高阶矩估计,在合成及宇宙学实验中提升了保留对数似然与校准。

摘要:

神经后验估计(NPE)将摊销条件密度估计器拟合到完整数据实现的模拟训练集。NPE的训练目标是对数似然,它倾向于为训练点分配任意高密度的后验近似值,而典型的摊销条件密度估计器族(如归一化流、高斯编码器)足够灵活以满足这一需求。由此产生的极端过自信策略对保留数据的泛化效果不佳。早停和其他正则化器可防止这种退化,但它们仅限于根据样本内残差设定的不利优化轨迹选择后验近似值,其中后验均值欠拟合而后验方差过拟合。为避免这种权衡,我们提出交叉拟合NPE(CF-NPE),它使用样本拆分将条件均值估计与后验高阶矩的估计分离。通过$K$折交叉拟合,CF-NPE拟合$K$个后验均值回归器,然后将条件密度估计器拟合到所得的折外残差。在合成数据实验和宇宙学应用中,CF-NPE比标准NPE实现了更高的保留对数似然,且通常改善了校准。CF-NPE在高维、模拟受限的设置中最有用。

英文摘要:

Neural Posterior Estimation (NPE) fits an amortized conditional density estimator to a simulated training set of complete-data realizations. NPE's training objective, log-likelihood, favors posterior approximations that assign arbitrarily high density to the training points, and typical families of amortized conditional density estimators (e.g., normalizing flows, Gaussian encoders) are flexible enough to oblige. The resulting policy of extreme overconfidence translates poorly to held-out data. Early stopping and other regularizers can prevent this degeneracy, but they are limited to selecting posterior approximations along an unfavorable optimization trajectory set by in-sample residuals, where the posterior mean is underfit while the posterior variance is overfit. To avoid this trade-off, we propose Cross-Fitting NPE (CF-NPE), which uses sample splitting to separate conditional-mean estimation from the estimation of the posterior's higher moments. Using $K$-fold cross-fitting, CF-NPE fits $K$ posterior-mean regressors and then fits a conditional density estimator to the resulting out-of-fold residuals. In experiments with synthetic data and in a cosmological application, CF-NPE achieves higher held-out log-likelihood than standard NPE and typically improves calibration. CF-NPE is most useful in high-dimensional, simulation-limited settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑