arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09959math.STcs.LGstat.MLstat.TH
收藏

通过调和分析学习多索引模型的统计-计算权衡

Statistical-Computational Trade-offs in Learning Multi-Index Models via Harmonic Analysis

Yale University(耶鲁大学)

Hugo Latourelle-Vigeant, Theodor Misiakiewicz

AI总结:

本文通过调和分析方法,研究多索引模型的学习复杂度权衡,提出频谱算法在样本和运行时间之间实现灵活的复杂度平衡。

摘要:

我们研究了学习多索引模型(MIMs)的问题,其中标签仅通过一个未知的s维投影W*^T x ∈ R^s依赖于输入x ∈ R^d。利用该问题在正交群O_d下的等变性,我们获得了具有球对称输入的MIMs的学习复杂度的精确调和-分析刻画——这细化并推广了之前针对高斯分布的具体分析。具体而言,我们推导了在统计查询(SQ)和低次多项式(LDP)框架内的统计和计算复杂度下界。这些下界自然地分解到球谐子空间中。受此分解的启发,我们构造了一族基于调和张量展开的频谱算法,依次恢复隐含方向并几乎达到这些SQ和LDP下界。根据谐波次数序列的选择,这些估计器可以实现样本复杂度和运行时间复杂度之间的广泛权衡。从技术角度看,我们的结果基于O_d作用在L^2(S^{d-1})上的半单分解以及球谐函数与迹零对称张量之间的 intertwining 同构关系。

英文摘要:

We study the problem of learning multi-index models (MIMs), where the label depends on the input $\boldsymbol{x} \in \mathbb{R}^d$ only through an unknown $\mathsf{s}$-dimensional projection $\boldsymbol{W}_*^\mathsf{T} \boldsymbol{x} \in \mathbb{R}^\mathsf{s}$. Exploiting the equivariance of this problem under the orthogonal group $\mathcal{O}_d$, we obtain a sharp harmonic-analytic characterization of the learning complexity for MIMs with spherically symmetric inputs -- which refines and generalizes previous Gaussian-specific analyses. Specifically, we derive statistical and computational complexity lower bounds within the Statistical Query (SQ) and Low-Degree Polynomial (LDP) frameworks. These bounds decompose naturally across spherical harmonic subspaces. Guided by this decomposition, we construct a family of spectral algorithms based on harmonic tensor unfolding that sequentially recover the latent directions and (nearly) achieve these SQ and LDP lower bounds. Depending on the choice of harmonic degree sequence, these estimators can realize a broad range of trade-offs between sample and runtime complexity. From a technical standpoint, our results build on the semisimple decomposition of the $\mathcal{O}_d$-action on $L^2 (\mathbb{S}^{d-1})$ and the intertwining isomorphism between spherical harmonics and traceless symmetric tensors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑