Optimal Posteriors for Chi-squared Divergence based PAC-Bayesian Bounds and Comparison with KL-divergence based Optimal Posteriors and Cross-Validation Procedure
Puja Sahu, Nandyala Hemachandra
备注：
arXiv admin note: text overlap with arXiv:1912.06803
We investigate optimal posteriors for recently introduced \cite{begin2016pac} chi-squared divergence based PAC-Bayesian bounds in terms of nature of their distribution, scalability of computations, and test set performance. For a finite classifier set, we deduce bounds for three distance functions: KL-divergence, linear and squared distances. Optimal posterior weights are proportional to deviations of empirical risks, usually with subset support. For uniform prior, it is sufficient to search among posteriors on classifier subsets ordered by these risks. We show the bound minimization for linear distance as a convex program and obtain a closed-form expression for its optimal posterior. Whereas that for squared distance is a quasi-convex program under a specific condition, and the one for KL-divergence is non-convex optimization (a difference of convex functions). To compute such optimal posteriors, we derive fast converging fixed point (FP) equations. We apply these approaches to a finite set of SVM regularization parameter values to yield stochastic SVMs with tight bounds. We perform a comprehensive performance comparison between our optimal posteriors and known KL-divergence based posteriors on a variety of UCI datasets with varying ranges and variances in risk values, etc. Chi-squared divergence based posteriors have weaker bounds and worse test errors, hinting at an underlying regularization by KL-divergence based posteriors. Our study highlights the impact of divergence function on the performance of PAC-Bayesian classifiers. We compare our stochastic classifiers with cross-validation based deterministic classifier. The latter has better test errors, but ours is more sample robust, has quantifiable generalization guarantees, and is computationally much faster.
On the Existence of Uniformly Most Powerful Bayesian Tests With Application to Non-Central Chi-Squared Tests
一致最大功效贝叶斯检验（Uniformly Most Powerful Bayesian Test，UMBBT）是贝叶斯假设检验的一个客观类别，可以被认为是经典一致最大功效检验的贝叶斯对应。因为UMPBT的拒绝区域可以与经典一致最大功效检验（UMPT）的拒绝区域相匹配，所以UMPBT提供了用于校准贝叶斯证据阈值、贝叶斯因子、经典显着性水平和p值的机制。本文的目的是将UMPBT的应用扩展到指数族模型之外。具体来说，我们引入了充分条件的存在UMBBT的，并提出了一个统一的方法，他们的推导。我们的方法的一个重要应用是扩展UMBBT的测试卡方分布的非中心性参数是否为零。由此产生的测试具有广泛的适用性，提供默认的替代假设来计算贝叶斯因子，例如，皮尔逊卡方检验的拟合优度，独立性测试的列联表，和似然比，得分和沃尔德测试。
Survival Prediction of Children Undergoing Hematopoietic Stem Cell Transplantation Using Different Machine Learning Classifiers by Performing Chi-squared Test and Hyper-parameter Optimization: A Retrospective Analysis
