基于广义拟梯度的稳健估计
Robust estimation via generalized quasi-gradients
摘要：
我们探讨了为什么许多最近提出的稳健估计问题是有效的解决，即使底层的优化问题是非凸的。我们研究这些稳健估计问题的损失景观，并确定“广义拟梯度”的存在。当这些准梯度存在时，保证一个大家族的低遗憾算法近似全局最小值;这包括常用的滤波算法。对于有界协方差下分布的稳健均值估计，证明了相关优化问题的任何一阶平稳点是{近似全局最小值}的充要条件是腐败水平$ε< 1/3$ .因此，任何接近平稳点的优化算法都会产生具有崩溃点$1/3$的有效鲁棒估计器。通过仔细的初始化和步长，我们将其改进为最佳的$1/2$。对于其他任务，包括线性回归和联合均值和协方差估计，损失情况更加崎岖：存在距离全局最小值任意远的稳定点。然而，我们证明了广义拟梯度的存在，并构建有效的算法。这些算法比以前的文献中，线性回归，我们提高了估计误差从$O(\sqrtε)$的最佳速率的$O(ε)$小$ε$假设认证的hypercontractivity。对于具有近似一致协方差的均值估计，我们证明了一个简单的梯度下降算法可以达到崩溃点$1/3$和迭代复杂度$\tilde{O}(d/ε^2)$。
英文摘要：
We explore why many recently proposed robust estimation problems are efficiently solvable, even though the underlying optimization problems are non-convex. We study the loss landscape of these robust estimation problems, and identify the existence of "generalized quasi-gradients". Whenever these quasi-gradients exist, a large family of low-regret algorithms are guaranteed to approximate the global minimum; this includes the commonly-used filtering algorithm. For robust mean estimation of distributions under bounded covariance, we show that any first-order stationary point of the associated optimization problem is an {approximate global minimum} if and only if the corruption level $ε< 1/3$. Consequently, any optimization algorithm that aproaches a stationary point yields an efficient robust estimator with breakdown point $1/3$. With careful initialization and step size, we improve this to $1/2$, which is optimal. For other tasks, including linear regression and joint mean and covariance estimation, the loss landscape is more rugged: there are stationary points arbitrarily far from the global minimum. Nevertheless, we show that generalized quasi-gradients exist and construct efficient algorithms. These algorithms are simpler than previous ones in the literature, and for linear regression we improve the estimation error from $O(\sqrtε)$ to the optimal rate of $O(ε)$ for small $ε$ assuming certified hypercontractivity. For mean estimation with near-identity covariance, we show that a simple gradient descent algorithm achieves breakdown point $1/3$ and iteration complexity $\tilde{O}(d/ε^2)$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
