arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.02154cond-mat.dis-nncond-mat.stat-mechstat.ML
收藏

临界渗透作为分析深度网络训练的框架

Critical Percolation as a Framework to Analyze the Training of Deep Networks

Zohar Ringel, Rodrigo de Bem

摘要:

在本文中,我们探讨了两个相关的深度学习主题:i)处理图结构的输入数据,ii)更好地理解和分析深度网络和相关的学习算法。考虑到这一点,我们专注于在一个特定的子集的平面图(迷宫)的可达性的拓扑分类。这样做,我们能够在保持在欧几里得空间的同时对数据的拓扑进行建模,从而允许使用标准的CNN架构进行处理。我们提出了一个合适的架构,这个问题,并表明它可以表达一个完美的解决方案的分类任务。这个解决方案的成本函数的形状也来自,值得注意的是,不依赖于大迷宫限制的迷宫的大小。造成这种行为的是数据集中的罕见事件,这些事件强烈地调节了这个全局最小值附近的成本函数的形状。我们进一步确定了学习的障碍,表现为表现不佳的局部极小值,其中网络选择忽略一些输入。我们进一步支持我们的主张与训练实验和数值分析的成本函数的网络上的$128$层。

英文摘要:

In this paper we approach two relevant deep learning topics: i) tackling of graph structured input data and ii) a better understanding and analysis of deep networks and related learning algorithms. With this in mind we focus on the topological classification of reachability in a particular subset of planar graphs (Mazes). Doing so, we are able to model the topology of data while staying in Euclidean space, thus allowing its processing with standard CNN architectures. We suggest a suitable architecture for this problem and show that it can express a perfect solution to the classification task. The shape of the cost function around this solution is also derived and, remarkably, does not depend on the size of the maze in the large maze limit. Responsible for this behavior are rare events in the dataset which strongly regulate the shape of the cost function near this global minimum. We further identify an obstacle to learning in the form of poorly performing local minima in which the network chooses to ignore some of the inputs. We further support our claims with training experiments and numerical analysis of the cost function on networks with up to $128$ layers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑