发表机构
Monash University; University of Haifa（莫纳什大学; 海法大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出稀疏斜规则提升方法，通过可学习稀疏线性变换扩展规则条件，使决策区域成为斜多面体，在14个任务上以更低模型复杂度保持或提升精度，兼顾可解释性与准确性。
AI 中文摘要
符号规则的小型加性集成提供了可解释的预测模型。传统上，这些集成使用基于单个输入变量 $x$ 和阈值 $t$ 的简单阈值命题 $x \geq t$ 的合取规则条件，在几何上产生轴平行多面体作为决策区域。虽然这种形式确保了个别规则的高度可解释性，并且可以使用梯度提升方法高效学习，但它依赖于访问一组精心设计的表达性输入特征，以便小型轴平行区域集成能够很好地描述目标变量。缺乏此类特征时，达到足够精度需要增加个别规则的数量和复杂性，这削弱了模型的可解释性。在此，我们通过引入具有可学习稀疏线性变换输入变量的逻辑命题来扩展经典规则集成，即形如 $\mathbf{x}^T\mathbf{w} \geq t$ 的命题，其中 $\mathbf{w}$ 是可学习的稀疏权重向量，使决策区域成为具有斜面的通用多面体。我们提出了一种基于加权逻辑回归的梯度提升学习方法。在14个回归和分类任务上的实证结果表明，所提出的方法在保持相似或更好预测精度的同时，实现了比竞争基线更低的模型复杂性。因此，该方法在可解释性和准确性之间提供了有利的权衡，并减少了对人工特征工程的依赖。
英文摘要
Small additive ensembles of symbolic rules offer interpretable prediction models. Traditionally, these ensembles use rule conditions based on conjunctions of simple threshold propositions $x \geq t$ on a single input variable $x$ and threshold $t$, resulting geometrically in axis-parallel polytopes as decision regions. While this form ensures a high degree of interpretability for individual rules and can be learned efficiently using the gradient boosting approach, it relies on having access to a curated set of expressive input features so that a small ensemble of axis-parallel regions can describe the target variable well. Absent such features, reaching sufficient accuracy requires increasing the number and complexity of individual rules, which diminishes the interpretability of the model. Here, we extend classical rule ensembles by introducing logical propositions with learnable sparse linear transformations of input variables, i.e., propositions of the form $\mathbf{x}^T\mathbf{w} \geq t$, where $\mathbf{w}$ is a learnable sparse weight vector, enabling decision regions as general polyhedrons with oblique faces. We propose a learning method using gradient boosting based on a weighted logistic regression. Empirical results across 14 regression and classification tasks demonstrate that the proposed method achieves lower model complexity than competitive baselines while maintaining similar or better predictive accuracy. Hence, the approach provides a favorable trade-off between interpretability and accuracy and reduces the reliance on manual feature engineering.
CommentsAccepted for publication in Journal of Data Mining and Knowledge Discovery. The Version of Record is forthcoming
