arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.09036math.STcs.LGecon.EMstat.MLstat.TH
收藏

正交统计学习

Orthogonal Statistical Learning

Microsoft Research(微软研究院);Stanford University(斯坦福大学)

Dylan J. Foster, Vasilis Syrgkanis

AI总结:

本文提出基于 Neyman 正交性与两阶段样本切分的统计学习框架,证明冗余参数估计误差对超额风险仅具二阶影响,并在复杂非参数设定下获得 oracle 速率。

摘要:

我们针对一种统计学习设定给出了非渐近超额风险保证:在该设定中,用于评估目标参数的总体风险依赖于一个必须从数据中估计的未知冗余参数。我们分析了一种两阶段样本切分元算法,该算法将用于目标参数和冗余参数的任意统计算法作为输入。我们证明,如果总体风险满足一个称为 Neyman 正交性的条件,则冗余参数估计误差对该元算法所获得的超额风险界的影响是二阶的。我们的定理不关心用于目标和冗余的具体算法,而只对它们各自的性能作出假设。这使得能够利用机器学习中的大量已有结果,为带有冗余成分的学习提供新的保证。此外,通过关注超额风险而非参数估计,我们能够在比以往工作更弱的假设下给出收敛速率,并适应目标参数属于复杂非参数类的设定。我们给出了关于冗余类和目标类度量熵的条件,使所得 oracle 速率与已知冗余参数时的速率具有相同阶数。

英文摘要:

We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input arbitrary estimation algorithms for the target parameter and nuisance parameter. We show that if the population risk satisfies a condition called Neyman orthogonality, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order. Our theorem is agnostic to the particular algorithms used for the target and nuisance and only makes an assumption on their individual performance. This enables the use of a plethora of existing results from machine learning to give new guarantees for learning with a nuisance component. Moreover, by focusing on excess risk rather than parameter estimation, we can provide rates under weaker assumptions than in previous works and accommodate settings in which the target parameter belongs to a complex nonparametric class. We provide conditions on the metric entropy of the nuisance and target classes such that oracle rates of the same order as if we knew the nuisance parameter are achieved.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑