arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.09912math.STstat.MLstat.TH
收藏

模型修复:过参数化统计模型的鲁棒恢复

Model Repair: Robust Recovery of Over-Parameterized Statistical Models

Chao Gao, John Lafferty

摘要:

介绍了一种新型的鲁棒估计问题,其目标是恢复一个统计模型,该模型已被损坏后,它已被估计的数据。提出了仅使用设计而不是用于在监督学习设置中拟合模型的响应值来“修复”模型的方法。理论的发展表明,两个重要的成分是必要的模型修复-统计模型必须过参数化,估计必须纳入冗余。特别是,估计的基础上随机梯度下降被认为是非常适合模型修复,但稀疏估计是不是在一般的可修复。在阐述了问题并建立了与抗差估计相关的关键技术引理之后,给出了一系列关于过参数化线性模型、随机特征模型和人工神经网络的修复结果。模拟研究证实和说明的理论研究结果。

英文摘要:

A new type of robust estimation problem is introduced where the goal is to recover a statistical model that has been corrupted after it has been estimated from data. Methods are proposed for "repairing" the model using only the design and not the response values used to fit the model in a supervised learning setting. Theory is developed which reveals that two important ingredients are necessary for model repair---the statistical model must be over-parameterized, and the estimator must incorporate redundancy. In particular, estimators based on stochastic gradient descent are seen to be well suited to model repair, but sparse estimators are not in general repairable. After formulating the problem and establishing a key technical lemma related to robust estimation, a series of results are presented for repair of over-parameterized linear models, random feature models, and artificial neural networks. Simulation studies are presented that corroborate and illustrate the theoretical findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑