arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1605.00329cs.LG
收藏

对神经网络几何和训练的一些见解

Some Insights into the Geometry and Training of Neural Networks

IBM Watson(IBM沃森)

Ewout van den Berg

摘要:

神经网络已经在快速增长的实际应用中成功地用于分类任务。尽管它们很受欢迎并被广泛使用,但仍然有许多方面的训练和分类没有得到很好的理解。在本文中,我们的目标是提供一些新的见解,通过分析神经网络从特征空间的角度来训练和分类。我们回顾并解释了决策区域的形成,并研究了它们的一些组合方面。我们特别强调神经网络的权重和偏置项之间的连接和属性的决策边界和其他地区表现出不同程度的分类信心。我们展示了如何在这些地区的错误backpropagates和强调的重要作用,他们在形成的梯度。这些发现揭示了权重参数的缩放与训练样本密度之间的联系。这揭示了消失梯度问题,解释了正则化的必要性,并提出了一种对训练数据进行二次采样以提高性能的方法。

英文摘要:

Neural networks have been successfully used for classification tasks in a rapidly growing number of practical applications. Despite their popularity and widespread use, there are still many aspects of training and classification that are not well understood. In this paper we aim to provide some new insights into training and classification by analyzing neural networks from a feature-space perspective. We review and explain the formation of decision regions and study some of their combinatorial aspects. We place a particular emphasis on the connections between the neural network weight and bias terms and properties of decision boundaries and other regions that exhibit varying levels of classification confidence. We show how the error backpropagates in these regions and emphasize the important role they have in the formation of gradients. These findings expose the connections between scaling of the weight parameters and the density of the training samples. This sheds more light on the vanishing gradient problem, explains the need for regularization, and suggests an approach for subsampling training data to improve performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑