深度网络的几何:幂图细分
The Geometry of Deep Networks: Power Diagram Subdivision
摘要:
我们研究了具有分段仿射和凸非线性的深度(神经)网络(DN)的几何结构。这种DN的层已经被证明是{ \em最大仿射样条算子}(MASO),其划分其输入空间并将区域相关仿射映射应用于其输入以产生其输出。我们证明了每个MASO层的输入空间分区对应于{ \em功率图}(经典Voronoi平铺的扩展),其区域数量相对于单元(神经元)的数量呈指数增长。我们进一步表明,MASO层的组合物(例如,整个DN)产生逐步细分的功率图并提供其分析形式。细分过程将仿射映射约束在(指数级多的)幂图区域上,以大大降低它们的复杂性。对于分类问题,我们得到一个公式的MASO DN的决策边界在输入空间加上一个衡量其曲率取决于DN的非线性,权重和架构。大量的数值实验支持和扩展了我们的理论结果。
英文摘要:
We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrate that each MASO layer's input space partitioning corresponds to a {\em power diagram} (an extension of the classical Voronoi tiling) with a number of regions that grows exponentially with respect to the number of units (neurons). We further show that a composition of MASO layers (e.g., the entire DN) produces a progressively subdivided power diagram and provide its analytical form. The subdivision process constrains the affine maps on the (exponentially many) power diagram regions to greatly reduce their complexity. For classification problems, we obtain a formula for a MASO DN's decision boundary in the input space plus a measure of its curvature that depends on the DN's nonlinearities, weights, and architecture. Numerous numerical experiments support and extend our theoretical results.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。