从尼桑自然证明出发的分布PAC学习
Distributional PAC-Learning from Nisan's Natural Proofs
AI总结:
本文研究自然证明与PAC学习之间的蕴含关系能否推广到更一般的电路类和分布PAC模型,提出了一种新的分布PAC学习框架,并基于通信复杂性类自然证明(包括Nisan证明)给出了深度为2多数电路、多面体和DNF的分布PAC学习算法。
摘要:
Carmosino等人(2016)证明了,针对$Λ$的电路下界自然证明可以推出学习$Λ$-电路的高效算法,但仅在均匀分布下、使用成员查询,并且需要满足$\AC^0[p] \subseteq Λ$。我们考虑这一蕴含关系能否推广到$Λ\not\supseteq \AC^0[p]$的情形,以及能否推广到仅使用随机样本并在任意样本分布上学习的学习算法(Valiant的PAC学习模型)。我们首先观察到,如果对于任意电路类$Λ$,都存在从$Λ$的自然证明到$Λ$的PAC学习的蕴含关系,那么基于格的密码学中的标准假设将不成立。特别地,我们观察到深度为2的多数电路是该蕴含关系的一个(条件性)反例,因为Nisan(1993)给出了一个自然证明,但Klivans和Sherstov(2009)在基于格的假设下证明了PAC学习的困难性。因此我们提出如下问题:从Nisan的自然证明中,我们可以合理地期望得到什么样的学习算法?我们的主要结果是,所有由一类通信复杂性论证产生的自然证明(包括Nisan的证明)都蕴含Valiant PAC模型的一种新的分布变体(即“平均情形”松弛)中的PAC学习算法。我们的分布PAC模型比Blum等人(1993)的平均情形预测模型和Nanashima(2021)的启发式PAC模型更强,并且具有若干使其具有独立研究价值的重要性质,例如具有提升友好性。我们结果的主要应用包括:在自然目标分布上针对深度为2的多数电路、多面体和DNF的新的分布PAC学习算法,以及不存在可由深度为2的多数电路求值的编码输入弱伪随机函数。
英文摘要:
Carmosino et al. (2016) demonstrated that natural proofs of circuit lower bounds for $Λ$ imply efficient algorithms for learning $Λ$-circuits, but only over \textit{the uniform distribution}, with \textit{membership queries}, and provided $\AC^0[p] \subseteq Λ$. We consider whether this implication can be generalized to $Λ\not\supseteq \AC^0[p]$, and to learning algorithms which use only random examples and learn over arbitrary example distributions (Valiant's PAC-learning model). We first observe that, if, for any circuit class $Λ$, there is an implication from natural proofs for $Λ$ to PAC-learning for $Λ$, then standard assumptions from lattice-based cryptography do not hold. In particular, we observe that depth-2 majority circuits are a (conditional) counter example to the implication, since Nisan (1993) gave a natural proof, but Klivans and Sherstov (2009) showed hardness of PAC-learning under lattice-based assumptions. We thus ask: what learning algorithms can we reasonably expect to follow from Nisan's natural proofs? Our main result is that all natural proofs arising from a type of communication complexity argument, including Nisan's, imply PAC-learning algorithms in a new \textit{distributional} variant (i.e., an ``average-case'' relaxation) of Valiant's PAC model. Our distributional PAC model is stronger than the average-case prediction model of Blum et al. (1993) and the heuristic PAC model of Nanashima (2021), and has several important properties which make it of independent interest, such as being \textit{boosting-friendly}. The main applications of our result are new distributional PAC-learning algorithms for depth-2 majority circuits, polytopes and DNFs over natural target distributions, as well as the nonexistence of encoded-input weak PRFs that can be evaluated by depth-2 majority circuits.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。