发表机构
Seminar for Statistics, ETH Zürich; IFZ, Lucerne University of Applied Sciences and Arts（统计系，苏黎世联邦理工学院; IFZ，卢塞恩应用科学与艺术大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对机器学习中隐藏混杂问题，为梯度提升开发非线性频谱去混杂框架，用频谱损失取代平方误差损失，通过频谱收缩与正则化相互作用实现去混杂，扩展方法到一般似然和非线性混杂，实验证明其能改善目标函数估计且更具可扩展性。
AI 中文摘要
灵活的机器学习方法可能对隐藏的混杂因素敏感：它们可能学习由未观察到的混杂因素引起的关联而非稳定信号。频谱去混杂通过收缩协变量矩阵的高方差方向来缓解此问题，在密集混杂下，这些方向携带潜在混杂因素信息。现有工作主要集中在线性模型。我们为梯度提升开发了一个非线性频谱去混杂框架。我们的方法用频谱损失取代普通平方误差损失，通过减缓在混杂对齐方向上的学习来改变提升动态。我们表明，去混杂不是仅由频谱损失实现的，而是通过频谱收缩和正则化之间的相互作用，特别是在早期停止方面。此外，我们提供了一个混合模型解释，将LAVA型收缩与随机效应调整联系起来，并产生了一个用于调整频谱损失的经验贝叶斯程序。我们还使用拉普拉斯近似和核随机效应将该方法扩展到一般似然和非线性混杂。在合成和实际实验中，频谱去混杂提升改善了隐藏混杂下目标函数的估计，并且比现有的非线性频谱去混杂基线更具可扩展性。
英文摘要
Flexible machine-learning methods can be sensitive to hidden confounding: they may learn associations induced by unobserved confounders rather than stable signals. Spectral deconfounding mitigates this problem by shrinking high-variance directions of the covariate matrix that, under dense confounding, carry latent confounder information. Existing work has largely focused on linear models. We develop a nonlinear spectral deconfounding framework for gradient boosting. Our approach replaces the ordinary squared-error loss by a spectral loss, which alters the boosting dynamics by slowing down learning in confounding-aligned directions. We show that deconfounding is not achieved by the spectral loss alone, but by the interaction between spectral shrinkage and regularization, especially in terms of early stopping. Moreover, we provide a mixed-model interpretation that connects LAVA-type shrinkage to random-effects adjustment and yields an empirical-Bayes procedure for tuning the spectral loss. We also extend the method to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects. Across synthetic and real-world experiments, spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is substantially more scalable than existing nonlinear spectral deconfounding baselines.
