AI 中文总结
研究在线主成分分析中相变的决定因素，以Oja算法为例，发现当$n,d\to\infty$且$n/d\log d\to\gamma\in(0,\infty)$时存在相变，临界状态下相关性随机，与普通高维PCA在恒定$n/d$时情况不同。
AI 中文摘要
高维统计理论已确立恒定纵横比的重要性，当维度数（$d$）和样本数（$n$）满足$n,d\to\infty$且$n/d\to\gamma\in(0,\infty)$时，有助于理解典型估计问题的极限。对于从$n$个独立同分布样本估计$d\times d$总体协方差矩阵的最大特征向量，BBP相变给出了精确阈值。本文研究在线/流算法，以Oja算法为例，在$n$个独立同分布样本$X_k\sim\mathcal{N}(0,\Sigma)$上运行，步长为$\delta/d$，输出$\hat v_n$。当$n,d\to\infty$且$n/d\log d\to\gamma\in(0,\infty)$时，建立了相变：当$\gamma<\gamma_*$时，$|\langle\hat v_n,v_0\rangle|\to 0$；当$\gamma>\gamma_*$时，$|\langle\hat v_n,v_0\rangle|\to\rho_*$。在临界状态下，相关性是随机的。这与普通高维主成分分析形成鲜明对比，普通情况下恒定$n/d$时可能有非零重叠且随$n/d$增加而改善。
英文摘要
High dimensional statistical theory has established the importance of constant aspect ratio, when the number of dimensions ($d$) and samples ($n$) satisfy $n,d\to\infty$ with $n/d\to γ\in(0,\infty)$, in understanding the limits of canonical estimation problems. In particular, for estimating the top eigenvector of a $d\times d$ population covariance matrix from $n$ iid samples, the BBP phase transition gives a precise threshold -- a simple functional of the aspect ratio -- such that the top sample principal component attains nonzero asymptotic correlation with the truth only when the leading population eigenvalue exceeds it. In this paper, we show that for online / streaming algorithms the story is very different, and constant aspect ratio is insufficient for nonzero overlap. We study Oja's algorithm, the most popular method for online PCA. Let $Σ=θ^2 v_0v_0^\top+I\in\mathbb{R}^{d\times d}$, and run Oja's algorithm with step size $δ/d$ on $n$ iid samples $X_k\sim\mathcal{N}(0,Σ)$, with output $\hat v_n$. Then, as $n,d\to\infty$ with $n/d\log d\toγ\in(0,\infty)$, we establish a phase transition: $|\langle\hat v_n,v_0\rangle|\to 0$ when $γ<γ_*$, and $\toρ_*$ when $γ>γ_*$. Here $ρ_*=ρ_*(θ,δ)=\sqrt{(θ^2-δ/2)_+/θ^2(1+δ/2)}$ and $γ_*=γ_*(θ,δ)=1/2δ(θ^2-δ/2)_+$. Further, at criticality, when $n=[γ_*d\log d+ηd]$ and $d\to\infty$, $η\in\mathbb{R}$, the correlation is random: $|\langle\hat v_n,v_0\rangle|\stackrel{w}{\to}ρ_*|G|\exp(η/2γ_*)/\sqrt{ρ_*^4+G^2\exp(η/γ_*)}$ where $G\sim\mathcal{N}(0,1)$. This is in stark contrast to ordinary high dimensional PCA, where nonzero overlap is possible at constant $n/d$ and improves as $n/d$ increases.
