arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.18147cs.LG
收藏

分而治之与合并的贝叶斯决策树采样

Divide, Conquer, Combine Bayesian Decision Tree Sampling

Jodie A. Cochrane, Adrian Wills, Sarah J. Johnson

AI总结:

本文提出DCC-Tree方法,通过将每个树结构与唯一决策参数关联,解决了贝叶斯决策树采样中结构与参数耦合难题,提升了采样效率与一致性。

摘要:

决策树因其灵活性和可解释性而成为常用的预测模型。本文旨在通过采用贝叶斯推断方法来量化决策树预测的不确定性。这具有挑战性,因为这些方法需要探索树结构空间以及与每个树结构相关的决策参数空间。这一问题已通过使用马尔可夫链蒙特卡洛(MCMC)方法加以处理,该方法构建一个马尔可夫链以从所需的贝叶斯估计中提供样本。重要的是,结构和决策参数紧密耦合;树结构的微小变化可能需要截然不同的决策参数才能提供准确的预测。现有MCMC方法面临的一个挑战是,提出在树结构和决策参数上的联合变化以实现高效采样。本文采用一种不同的方法,将每个不同的树结构与一组唯一的决策参数相关联。所提出的方法名为DCC-Tree,其灵感来源于Zhou等人[23]在概率程序方面的工作以及Cochrane等人[4]在基于哈密顿蒙特卡洛(HMC)的决策树采样方面的工作。结果表明,DCC-Tree的性能与其他基于HMC的方法相当,并且优于现有的贝叶斯树方法,同时在一致性方面有所改善并降低了每次提议的复杂度。

英文摘要:

Decision trees are commonly used predictive models due to their flexibility and interpretability. This paper is directed at quantifying the uncertainty of decision tree predictions by employing a Bayesian inference approach. This is challenging because these approaches need to explore both the tree structure space and the space of decision parameters associated with each tree structure. This has been handled by using Markov Chain Monte Carlo (MCMC) methods, where a Markov Chain is constructed to provide samples from the desired Bayesian estimate. Importantly, the structure and the decision parameters are tightly coupled; small changes in the tree structure can demand vastly different decision parameters to provide accurate predictions. A challenge for existing MCMC approaches is proposing joint changes in both the tree structure and the decision parameters that result in efficient sampling. This paper takes a different approach, where each distinct tree structure is associated with a unique set of decision parameters. The proposed approach, entitled DCC-Tree, is inspired by the work in Zhou et al. [23] for probabilistic programs and Cochrane et al. [4] for Hamiltonian Monte Carlo (HMC) based sampling for decision trees. Results show that DCC-Tree performs comparably to other HMC-based methods and better than existing Bayesian tree methods while improving on consistency and reducing the per-proposal complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑