arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.01394stat.MLcs.DScs.LG
收藏

稳健线性回归:多项式时间内的最优速率

Robust Linear Regression: Optimal Rates in Polynomial Time

CMU(卡内基梅隆大学)

Ainesh Bakshi, Adarsh Prasad

摘要:

我们获得了强大的和计算有效的估计学习几个线性模型,达到统计最优的收敛速度下最小的分布假设。具体地说,我们假设我们的数据是从一个$k$ -hypercontractive分布和一个$ε$ -分数是adversarially损坏。然后,我们描述了一个估计,收敛到最佳的最小二乘最小化的真实分布的速度成正比的$ε^{2-2/k}$,当噪声是独立的协变量。我们注意到,在我们的工作之前,没有这样的估计是已知的,即使使用无界计算。我们实现的速率是信息理论上最优的,因此我们解决了Klivans,Kothari和Meka [COLT'18]中的主要开放问题。我们的关键见解是确定一个分析条件,作为一个多项式松弛的独立性的随机变量。特别是,我们表明,当噪声和协变量的时刻是负相关的,我们得到了独立的噪声相同的速率。此外,当条件不满足时,我们获得与$ε^{2-4/k}$成比例的速率,并且再次匹配信息理论下限。我们的核心技术贡献是算法上利用独立的随机变量的“平方和”的框架,制定为上述多项式不等式。

英文摘要:

We obtain robust and computationally efficient estimators for learning several linear models that achieve statistically optimal convergence rate under minimal distributional assumptions. Concretely, we assume our data is drawn from a $k$-hypercontractive distribution and an $ε$-fraction is adversarially corrupted. We then describe an estimator that converges to the optimal least-squares minimizer for the true distribution at a rate proportional to $ε^{2-2/k}$, when the noise is independent of the covariates. We note that no such estimator was known prior to our work, even with access to unbounded computation. The rate we achieve is information-theoretically optimal and thus we resolve the main open question in Klivans, Kothari and Meka [COLT'18]. Our key insight is to identify an analytic condition that serves as a polynomial relaxation of independence of random variables. In particular, we show that when the moments of the noise and covariates are negatively-correlated, we obtain the same rate as independent noise. Further, when the condition is not satisfied, we obtain a rate proportional to $ε^{2-4/k}$, and again match the information-theoretic lower bound. Our central technical contribution is to algorithmically exploit independence of random variables in the "sum-of-squares" framework by formulating it as the aforementioned polynomial inequality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑