Axis-aligned decision trees are fast and stable but struggle on datasets with rotated or interaction-dependent decision boundaries, where informative splits require linear combinations of features rather than single-feature thresholds. Oblique forests address this with per-node hyperplane splits, but at added computational cost and implementation complexity. We propose a simple alternative: JARF, Jacobian-Aligned Random Forests. Concretely, we first fit an axis-aligned forest to estimate class probabilities or regression outputs, compute finite-difference gradients of these predictions with respect to each feature, aggregate them into an expected Jacobian outer product that generalizes the expected gradient outer product (EGOP), and use it as a single global linear preconditioner for all inputs. This supervised preconditioner applies a single global rotation of the feature space, then hands the transformed data back to a standard axis-aligned forest, preserving off-the-shelf training pipelines while capturing oblique boundaries and feature interactions that would otherwise require many axis-aligned splits to approximate. The same construction applies to any model that provides gradients, though we focus on random forests and gradient-boosted trees in this work. On tabular classification and regression benchmarks, this preconditioning consistently improves axis-aligned forests and often matches or surpasses oblique baselines while improving training time. Our experimental results and theoretical analysis together indicate that supervised preconditioning can recover much of the accuracy of oblique forests while retaining the simplicity and robustness of axis-aligned trees.
Generalized Random Forests using Fixed-Point Trees
Department of Mathematics and Statistics, McGill University, Montreal, Canada（数学与统计学系，麦吉尔大学，加拿大蒙特利尔）；Mila - Quebec AI Institute, Montreal, Quebec, Canada（魁北克AI研究院，加拿大蒙特利尔）
Stochastic geometry to generalize the Mondrian Process
STIT（stable under iterated tessellation）过程是一个随机过程，它产生一个递归的空间划分，其切割方向独立于球面上的分布。随机轴对齐切割的情况被称为蒙德里安过程。随机森林和拉普拉斯核近似建立从蒙德里安过程导致了有效的在线学习方法和贝叶斯优化。在这项工作中，我们利用随机几何的工具来解决一些基本问题，STIT过程中的机器学习。首先，我们表明，STIT过程与切割方向绘制从离散分布可以有效地模拟提升到一个更高的维轴对齐的蒙德里安过程。其次，我们描述了所有可能的内核，平稳STIT过程和它们的混合物可以近似。我们还给出了STIT核对目标核的逼近误差的一致收敛速度，推广了[3]在Mondrian情况下的工作。第三，我们得到的一致性结果STIT森林的密度估计和回归。最后，我们给出了一个公式的密度估计所产生的一个无限的STIT随机森林。这允许在密度估计中的蒙德里安森林、蒙德里安核和拉普拉斯核之间进行精确比较。我们的论文呼吁在随机几何和机器学习的新交叉点上进一步发展。
Can synthetic data reproduce real-world findings in epidemiology? A replication study using adversarial random forests
Leibniz Institute for Prevention Research and Epidemiology—BIPS（预防研究与流行病学研究所—BIPS）；Faculty of Mathematics and Computer Science, University of Bremen（不莱梅大学数学与计算机科学学院）；Department of Human Health and Nutritional Sciences, University of Guelph（圭尔夫大学人类健康与营养科学系）；Institute of Epidemiology and Social Medicine, University of Münster（明斯特大学流行病学与社会医学研究所）；Institute of General Practice/Family Medicine, Faculty of Medicine and Medical Center, University of Freiburg（弗赖堡大学内科/家庭医学研究所）；Freiburg Center for Data Analysis, Modeling and AI, University of Freiburg（弗赖堡数据分析、建模与人工智能中心）；German Cancer Research Center (DKFZ), Heidelberg, Germany（德国癌症研究中心（DKFZ））；Department of Epidemiology, Helmholtz Centre for Infection Research (HZI), Brunswick, Germany（感染研究哈勒研究所流行病学部）；Department of Epidemiology and Preventive Medicine, University of Regensburg（雷根施塔特大学流行病学与预防医学系）；Institute for Occupational and Maritime Medicine (ZfAM), University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心职业与航海医学研究所）；Saarland Cancer Registry, Saarbrücken, Germany（萨尔州癌症登记处）
