arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.00515stat.ME
收藏

用于非参数回归的贝叶斯神经树模型

Bayesian Neural Tree Models for Nonparametric Regression

Tanujit Chakraborty, Gauri Kamat, Ashis Kumar Chakraborty

AI总结:

本研究针对非参数回归问题,提出融合贝叶斯与频率学派方法的贝叶斯神经树(BNT)模型,该模型参数量少、可同步特征选择与预测,在小样本下泛化性好,还推导了最优参数并通过真实数据集验证。

摘要:

频率学派与贝叶斯学派方法在诸多方面存在差异,但具备一些基本的最优性质。在实际的分类与回归问题中,常会出现基于某种主观准则,更偏好采用其中一类方法构建模型的情况。决策树、神经网络这类非参数分类与回归技术,既有从频率学派视角出发的学习方法(如分类与回归树(CART)、人工神经网络),也有贝叶斯学派的对应方法(如贝叶斯CART、贝叶斯神经网络)。本研究提出两种融合CART与神经网络的贝叶斯、频率学派版本的混合模型,我们将其命名为贝叶斯神经树(BNT)模型。两类模型均利用了决策树的架构,且需调优的参数数量少于先进神经网络。这类模型可同时执行特征选择与预测任务,灵活性极高,在训练观测样本数量有限的场景下泛化性能良好。我们研究了所提模型的一致性,并推导了一个关键模型参数的最优取值。此外,我们还利用多种真实回归数据集给出了示例验证。

英文摘要:

Frequentist and Bayesian methods differ in many aspects, but share some basic optimal properties. In real-life classification and regression problems, situations exist in which a model based on one of the methods is preferable based on some subjective criterion. Nonparametric classification and regression techniques, such as decision trees and neural networks, have frequentist (classification and regression trees (CART) and artificial neural networks) as well as Bayesian (Bayesian CART and Bayesian neural networks) approaches to learning from data. In this work, we present two hybrid models combining the Bayesian and frequentist versions of CART and neural networks, which we call the Bayesian neural tree (BNT) models. Both models exploit the architecture of decision trees and have lesser number of parameters to tune than advanced neural networks. Such models can simultaneously perform feature selection and prediction, are highly flexible, and generalize well in settings with a limited number of training observations. We study the consistency of the proposed models, and derive the optimal value of an important model parameter. We also provide illustrative examples using a wide variety of real-life regression data sets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑