arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1905.11345cs.LGcs.ITmath.ITstat.ML
收藏

分形的神经网络函数表示

Expression of Fractals Through Neural Network Functions

Duke University(杜克大学)

Nadav Dym, Barak Sober, Ingrid Daubechies

摘要:

为了帮助理解神经网络(NN)的潜在机制,近年来,一些研究小组研究了深度神经网络(DNN)生成的分段线性函数的线性区域$\ell$的数量。特别是,他们发现$\ell$可以随着网络参数$p$的数量呈指数增长,这一特性通常用于解释DNN在逼近复杂函数方面优于浅NN的优势。尽管如此,一个简单的维数参数表明,DNN不能像$\ell > p$那样快地生成具有$\ell$线性区域的所有分段线性函数。因此,寻求用可以由DNN构造的$\ell$线性区域来表征特定的函数族是很自然的。迭代函数系统(IFS)生成分段线性函数序列$F_k$,其线性区域的数量在$k$中是指数的。我们表明,在温和的假设下,$F_k$可以产生的NN只使用$\mathcal{O}(k)$参数。IFS被广泛用于以低计算成本在人工图像中生成看起来自然的景观纹理。它们也被提出用于压缩自然图像,尽管商业上的成功较少。这种基于分形的压缩令人惊讶的良好性能表明,我们的视觉系统可能会在某种程度上锁定图像中的自相似性。这种现象与DNN有效逼近IFS的能力相结合,可能有助于DNN的成功,特别是对于图像处理任务,以及提出基于DNN机制表示图像中自相似性的新算法。

英文摘要:

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network parameters $p$, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with $\ell$ linear regions as soon as $\ell > p$. It is thus natural to seek to characterize specific families of functions with $\ell$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions $F_k$ with a number of linear regions exponential in $k$. We show that, under mild assumptions, $F_k$ can be generated by a NN using only $\mathcal{O}(k)$ parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑