For multiple reasons -- such as avoiding overtraining from one data set or because of having received numerical estimates for some parameters in a model from an alternative source -- it is sometimes useful to divide a model's parameters into one group of primary parameters and one group of nuisance parameters. However, uncertainty in the values of nuisance parameters is an inevitable factor that impacts the model's reliability. This paper examines the issue of uncertainty calculation for primary parameters of interest in the presence of nuisance parameters. We illustrate a general procedure on two distinct model forms: 1) the GARCH time series model with univariate nuisance parameter and 2) multiple hidden layer feed-forward neural network models with multivariate nuisance parameters. Leveraging an existing theoretical framework for nuisance parameter uncertainty, we show how to modify the confidence regions for the primary parameters while considering the inherent uncertainty introduced by nuisance parameters. Furthermore, our study validates the practical effectiveness of adjusted confidence regions that properly account for uncertainty in nuisance parameters. Such an adjustment helps data scientists produce results that more honestly reflect the overall uncertainty.
Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference
Carnegie Mellon University(卡内基梅隆大学);Università di Padova(帕多瓦大学);Istituto Nazionale di Fisica Nucleare(国家核物理研究所);Lulea Techniska Universitet(吕勒奥理工大学);Universal Scientific Education and Research Network(环球科学教育与研究网络);Universidade Federal de São Carlos(圣卡洛斯联邦大学)
Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters
CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);University College London(伦敦大学学院);Inria(法国国家信息与自动化研究所);Gatsby Computational Neuroscience Unit(盖茨计算神经科学单元);University of Adelaide(阿德莱德大学)