低次学习与多项式的度量熵
Low-degree learning and the metric entropy of polynomials
AI总结:
本文通过证明低次多项式类的 L2 填充数双边估计,确立了学习该类所需查询复杂度的精确下界,并给出 Fourier 谱集中于少数子集的近似多项式类的随机查询复杂度上界及其在近似 juntas、常数深度电路等学习问题中的应用。
摘要:
设 $\mathscr{F}_{n,d}$ 为 $n$ 维离散超立方体 $\{-1,1\}^n$ 上所有次数至多为 $d$ 的函数 $f:\{-1,1\}^n\to[-1,1]$ 构成的类。在本文第一部分中,我们证明任何以 $L_2$ 精度 $\varepsilon$ 学习 $\mathscr{F}_{n,d}$ 的(确定性或随机化)算法,对于足够大的 $n$,至少需要 $\Omega((1-\sqrt{\varepsilon})2^d\log n)$ 次查询,从而确立了 Eskenazis 和 Ivanisvili (2021) 最近给出的上界在 $n\to\infty$ 时的精确性。为此,我们证明概念类 $\mathscr{F}_{n,d}$ 的 $L_2$-填充数 $\mathsf{M}(\mathscr{F}_{n,d},\\|\cdot\\|_{L_2},\varepsilon)$ 满足双边估计 $$c(1-\varepsilon)2^d\log n \leq \log \mathsf{M}(\mathscr{F}_{n,d},\\|\cdot\\|_{L_2},\varepsilon) \leq \frac{2^{Cd}\log n}{\varepsilon^4}$$ 其中 $n$ 足够大,$c, C>0$ 为通用常数。在本文第二部分中,我们对 Fourier 谱集中于少数子集的有界近似多项式类的随机化查询复杂度给出了对数上界。作为一个应用,我们证明了学习给定次数的近似 juntas、Fourier 尾部快速衰减的函数以及给定规模的常数深度电路所需随机查询数量的新估计。最后,我们得到了在查询模型和随机示例模型中无误差学习多项式类 $\mathscr{F}_{n,d}$ 所需查询数量的界。
英文摘要:
Let $\mathscr{F}_{n,d}$ be the class of all functions $f:\{-1,1\}^n\to[-1,1]$ on the $n$-dimensional discrete hypercube of degree at most $d$. In the first part of this paper, we prove that any (deterministic or randomized) algorithm which learns $\mathscr{F}_{n,d}$ with $L_2$-accuracy $\varepsilon$ requires at least $Ω((1-\sqrt{\varepsilon})2^d\log n)$ queries for large enough $n$, thus establishing the sharpness as $n\to\infty$ of a recent upper bound of Eskenazis and Ivanisvili (2021). To do this, we show that the $L_2$-packing numbers $\mathsf{M}(\mathscr{F}_{n,d},\|\cdot\|_{L_2},\varepsilon)$ of the concept class $\mathscr{F}_{n,d}$ satisfy the two-sided estimate $$c(1-\varepsilon)2^d\log n \leq \log \mathsf{M}(\mathscr{F}_{n,d},\|\cdot\|_{L_2},\varepsilon) \leq \frac{2^{Cd}\log n}{\varepsilon^4}$$ for large enough $n$, where $c, C>0$ are universal constants. In the second part of the paper, we present a logarithmic upper bound for the randomized query complexity of classes of bounded approximate polynomials whose Fourier spectra are concentrated on few subsets. As an application, we prove new estimates for the number of random queries required to learn approximate juntas of a given degree, functions with rapidly decaying Fourier tails and constant depth circuits of given size. Finally, we obtain bounds for the number of queries required to learn the polynomial class $\mathscr{F}_{n,d}$ without error in the query and random example models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。