arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.01403hep-thcond-mat.dis-nncs.LGstat.ML
收藏

Nonperturbative renormalization for the neural network-QFT correspondence

Massachusetts Institute of Technology(麻省理工学院);NSF AI Institute for Artificial Intelligence and Fundamental Interactions(美国国家科学基金会人工智能与基础相互作用研究所);Université Paris Saclay(巴黎萨克雷大学);CEA(法国原子能和替代能源委员会);LIST(法国原子能和替代能源委员会实验室集成系统与技术研究所);Université d’Abomey-Calavi(阿波美-卡拉维大学)

Harold Erbin, Vincent Lahoche, Dine Ousmane Samary

英文摘要:

In a recent work arXiv:2008.08601, Halverson, Maiti and Stoner proposed a description of neural networks in terms of a Wilsonian effective field theory. The infinite-width limit is mapped to a free field theory, while finite $N$ corrections are taken into account by interactions (non-Gaussian terms in the action). In this paper, we study two related aspects of this correspondence. First, we comment on the concepts of locality and power-counting in this context. Indeed, these usual space-time notions may not hold for neural networks (since inputs can be arbitrary), however, the renormalization group provides natural notions of locality and scaling. Moreover, we comment on several subtleties, for example, that data components may not have a permutation symmetry: in that case, we argue that random tensor field theories could provide a natural generalization. Second, we improve the perturbative Wilsonian renormalization from arXiv:2008.08601 by providing an analysis in terms of the nonperturbative renormalization group using the Wetterich-Morris equation. An important difference with usual nonperturbative RG analysis is that only the effective (IR) 2-point function is known, which requires setting the problem with care. Our aim is to provide a useful formalism to investigate neural networks behavior beyond the large-width limit (i.e.~far from Gaussian limit) in a nonperturbative fashion. A major result of our analysis is that changing the standard deviation of the neural network weight distribution can be interpreted as a renormalization flow in the space of networks. We focus on translations invariant kernels and provide preliminary numerical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑