AI 中文总结
该研究提出分裂候选数量作为梯度提升决策树（GBDTs）的单轴容量参数，通过分析其与双重下降现象的关联，经XGBoost等模型实验验证了分裂候选预算对测试误差的影响规律。
AI 中文摘要
双重下降通常通过缩放显式容量参数（如神经网络宽度）进行研究。然而，对于梯度提升决策树（GBDTs），尚未建立类似的单轴容量参数。我们提出分裂候选数量作为GBDTs的可操作容量参数。在保持其他训练控制固定的情况下，增加分裂候选预算会细化特征量化网格，并扩充提升算法选择更新的根到叶路径字典。为分析这种扩充，我们构建了经验树核诊断，总结候选诱导路径如何分组训练样本。当经验核秩增长到接近样本量且出现极小正特征值时，会暴露出对噪声敏感的方向；在此 regime 中，测试误差先达到峰值，随后在更大的分裂候选预算下再次下降。该视角预测：更深的树应使用更少的分裂候选达到该 regime，更大的训练集应需要更精细的网格，标签噪声会使峰值更明显。实验支持这些预测，结果显示在XGBoost、LightGBM和CatBoost中，测试误差在中间分裂候选预算处出现峰值，而随机森林对照在相同分裂候选扫描下单调提升。综上，我们的分析和实验支持分裂候选缩放作为研究GBDTs的单轴容量干预方法，并表明观察到的双重下降源于候选诱导几何与提升动力学之间的相互作用。
英文摘要
Double descent is commonly studied by scaling an explicit capacity parameter, such as neural-network width. For gradient boosting decision trees (GBDTs), however, an analogous single-axis capacity parameter has not been established. We propose the number of split candidates as an operational capacity parameter for GBDTs. Holding other training controls fixed, increasing the split-candidate budget refines the feature-quantization grid and expands the dictionary of root-to-leaf paths from which boosting selects its updates. To analyze this expansion, we construct an empirical tree-kernel diagnostic that summarizes how candidate-induced paths group the training examples. A regime in which the empirical kernel rank grows toward the sample size and very small positive eigenvalues emerge exposes noise-sensitive directions; in this regime, test error peaks before decreasing again at larger split-candidate budgets. This perspective predicts that deeper trees should reach the regime with fewer split candidates, larger training sets should require finer grids, and label noise should make the peak more pronounced. Experiments support these predictions and show test-error peaks at intermediate split-candidate budgets across XGBoost, LightGBM, and CatBoost, whereas a random-forest control improves monotonically under the same split-candidate sweep. Taken together, our analysis and experiments support split-candidate scaling as a single-axis capacity intervention for studying GBDTs and suggest that the observed double descent arises from an interaction between candidate-induced geometry and boosting dynamics.
