发表机构
University of Pisa; ISTI-CNR（比萨大学; ISTI-CNR）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于TTVAE的TREVIS方法，将决策树映射至潜在空间实现梯度优化，所获决策树兼具与近优算法相当的预测性能及更优结构稀疏性。
AI 中文摘要
决策树是机器学习中应用最广泛的模型之一，很大程度上得益于其透明的决策逻辑，使其非常适合高风险决策场景。然而，大多数现有学习算法都聚焦于预测性能，忽略了其他理想属性的联合优化，例如结构稀疏性。在本研究中，我们提出了TREVIS，一种基于探索树Transformer变分自编码器（Tree Transformer Variational Auto-Encoder，简称TTVAE）的潜在空间来学习满足复杂目标的决策树的方法。通过将决策树映射到潜在表示，TREVIS将离散搜索空间替换为连续空间，从而能够通过可微代理模型进行基于梯度的优化。我们将TREVIS应用于学习同时优化预测性能和稀疏性的决策树，实验结果表明，TREVIS发现的决策树达到了现有近优算法的预测性能，同时提升了其结构稀疏性。
英文摘要
Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity. In this work we propose TREVIS, an approach for learning decision trees with respect to complex objectives, based on the exploration of the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). By mapping decision trees onto latent representations, TREVIS replaces the discrete search space with a continuous one, enabling gradient-based optimization via a differentiable surrogate model. We experiment with TREVIS for learning decision trees that jointly optimize predictive performance and sparsity. Results show that TREVIS discovers decision trees matching the predictive performance of existing near-optimal algorithms while improving their structural sparsity.
CommentsAccepted for publication at the 2026 IEEE International Conference on Data Mining (ICDM 2026)
