RS-ORT：一种用于最优回归树的空间缩减分支定界算法
RS-ORT: A Reduced-Space Branch-and-Bound Algorithm for Optimal Regression Trees
AI总结：
针对现有回归树MIP方法处理连续大规模数据的难题，提出仅在树结构变量分支的RS-ORT分支定界算法，结合多种界收紧技术与并行策略，在百万级数据集上实现更优性能与泛化能力。
摘要：
混合整数规划（MIP）已成为学习最优决策树的强大框架。然而，现有的回归任务MIP方法要么仅适用于纯二值特征，要么在处理连续型大规模数据时计算上难以处理。对连续特征进行朴素二值化会牺牲全局最优性，且往往生成深度不必要的树。我们将最优回归树训练重构为两阶段优化问题，并提出空间缩减最优回归树（RS-ORT）——一种仅在树结构变量上分支的专用分支定界（BB）算法。该设计保证了算法的收敛性，且其性能不受训练样本数量的影响。利用模型的结构特性，我们引入了多种界收紧技术——闭式叶节点预测、经验阈值离散化和精确深度1子树解析，这些技术与可分解的上下界策略相结合以加速训练。BB的节点级分解可实现简易的并行执行，进一步缓解了即使是百万级数据集的计算难题。基于对多个同时包含二值和连续特征的回归基准的实证研究，RS-ORT在训练和测试性能上也优于当前最优方法。值得注意的是，在包含多达2000000个带连续特征样本的数据集上，RS-ORT可在4小时内获得有保证的训练性能，同时具备更简单的树结构和更好的泛化能力。
英文摘要：
Mixed-integer programming (MIP) has emerged as a powerful framework for learning optimal decision trees. Yet, existing MIP approaches for regression tasks are either limited to purely binary features or become computationally intractable when continuous, large-scale data are involved. Naively binarizing continuous features sacrifices global optimality and often yields needlessly deep trees. We recast the optimal regression-tree training as a two-stage optimization problem and propose Reduced-Space Optimal Regression Trees (RS-ORT) - a specialized branch-and-bound (BB) algorithm that branches exclusively on tree-structural variables. This design guarantees the algorithm's convergence and its independence from the number of training samples. Leveraging the model's structure, we introduce several bound tightening techniques - closed-form leaf prediction, empirical threshold discretization, and exact depth-1 subtree parsing - that combine with decomposable upper and lower bounding strategies to accelerate the training. The BB node-wise decomposition enables trivial parallel execution, further alleviating the computational intractability even for million-size datasets. Based on the empirical studies on several regression benchmarks containing both binary and continuous features, RS-ORT also delivers superior training and testing performance than state-of-the-art methods. Notably, on datasets with up to 2,000,000 samples with continuous features, RS-ORT can obtain guaranteed training performance with a simpler tree structure and a better generalization ability in four hours.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
