arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.06100cs.LGcs.DSstat.ML
收藏

后选择泛化的局限性

The Limits of Post-Selection Generalization

Georgetown University(乔治城大学);Boston University(波士顿大学);IBM Research – Almaden(IBM阿尔马登研究中心);Weizmann Institute of Science(魏茨曼科学研究所);Northeastern University(东北大学)

Kobbi Nissim, Adam Smith, Thomas Steinke, Uri Stemmer, Jonathan Ullman

AI总结:

本研究揭示了满足事后泛化的算法的局限性,给出了其回答自适应统计查询的误差紧下界,且证明该性质不满足组合封闭性。

摘要:

尽管统计学和机器学习提供了大量确保泛化的方法,但这些方法在存在适应性时往往失效——适应性是指分析的选择取决于与同一数据集的先前交互的常见做法。近期一系列工作引入了强大的通用算法,可确保事后泛化(也称为鲁棒泛化或后选择泛化),该性质表明,给定算法的输出,很难找到任何统计量使得数据与其来源的总体存在显著差异。本研究展示了满足事后泛化的算法的若干局限性:首先,我们对任何满足事后泛化且回答自适应选择的统计查询的算法的误差给出了紧下界,显示了后选择数据分析进展的强大障碍;其次,我们证明事后泛化不满足组合封闭性,尽管许多此类算法表现出强组合性质。

英文摘要:

While statistics and machine learning offer numerous methods for ensuring generalization, these methods often fail in the presence of adaptivity---the common practice in which the choice of analysis depends on previous interactions with the same dataset. A recent line of work has introduced powerful, general purpose algorithms that ensure post hoc generalization (also called robust or post-selection generalization), which says that, given the output of the algorithm, it is hard to find any statistic for which the data differs significantly from the population it came from. In this work we show several limitations on the power of algorithms satisfying post hoc generalization. First, we show a tight lower bound on the error of any algorithm that satisfies post hoc generalization and answers adaptively chosen statistical queries, showing a strong barrier to progress in post selection data analysis. Second, we show that post hoc generalization is not closed under composition, despite many examples of such algorithms exhibiting strong composition properties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑