关于双鞍点与真凸性的间隙：特征向量逼近的一个Omega（log d）下界
On the Gap Between Strict-Saddles and True Convexity: An Omega(log d) Lower Bound for Eigenvector Approximation
摘要：
我们证明了一个\emph{query complexity}下界秩一主成分分析（PCA）。我们考虑一个预言机模型，给定一个对称矩阵$M \in \mathbb{R}^{d \times d}$，允许一个算法，使$T$ \emph{exact}查询的形式$w^{(i)} = Mv^{(i)}$的$i \in \{1,\dots,T\}$，其中$v^{(i)}$是从一个分布，任意依赖于过去的查询和测量$\{v^{(j)},w^{(j)}\}_{1 \le j \le i-1}$。我们表明，对于一个小的常数$ε$，任何自适应，随机算法，它可以找到一个单位向量$\widehat{v}$的$\widehat{v}^{\top}M\widehat{v} \ge (1-ε)\|M\|$，即使是小概率，必须使$T = Ω(\log d)$查询。除了解决一个广泛持有的民间猜想，这个界限证明了凸优化和“严格鞍”非凸优化之间的根本差距，PCA是一个典型的例子：在前者中，一阶方法可以有无量纲迭代复杂性，而在PCA中，基于梯度的方法的迭代复杂性必须随着维数的增加而增长。我们的论点通过减少估计的秩一穗变形维格纳模型。我们建立了这个模型的下限，通过开发一个“截断”类似的$χ^2$贝叶斯风险下限陈等人。
英文摘要：
We prove a \emph{query complexity} lower bound on rank-one principal component analysis (PCA). We consider an oracle model where, given a symmetric matrix $M \in \mathbb{R}^{d \times d}$, an algorithm is allowed to make $T$ \emph{exact} queries of the form $w^{(i)} = Mv^{(i)}$ for $i \in \{1,\dots,T\}$, where $v^{(i)}$ is drawn from a distribution which depends arbitrarily on the past queries and measurements $\{v^{(j)},w^{(j)}\}_{1 \le j \le i-1}$. We show that for a small constant $ε$, any adaptive, randomized algorithm which can find a unit vector $\widehat{v}$ for which $\widehat{v}^{\top}M\widehat{v} \ge (1-ε)\|M\|$, with even small probability, must make $T = Ω(\log d)$ queries. In addition to settling a widely-held folk conjecture, this bound demonstrates a fundamental gap between convex optimization and "strict-saddle" non-convex optimization of which PCA is a canonical example: in the former, first-order methods can have dimension-free iteration complexity, whereas in PCA, the iteration complexity of gradient-based methods must necessarily grow with the dimension. Our argument proceeds via a reduction to estimating the rank-one spike in a deformed Wigner model. We establish lower bounds for this model by developing a "truncated" analogue of the $χ^2$ Bayes-risk lower bound of Chen et al.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
