带随机高斯权重的深度神经网络:一种通用分类策略?"
Comments on "Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?"
摘要:
在最近发表的一篇论文[1]中,表明具有随机高斯权重的深度神经网络(DNN)保留了数据的度量结构,具有当两个数据点之间的角度较小时距离缩小更多的特性。我们同意[1]中考虑的随机投影设置以高概率保持距离。但就我们而言,数据点的角度和输出距离之间的关系正好相反,即,较小的角度导致较弱的距离收缩。这使我们得出结论,[1]中的定理3和图5是不准确的.因此,在DNN中使用随机高斯权重不能提供通用分类或单独处理类内和类外数据的能力。因此,仅由随机高斯权重组成的网络的行为对于解释DNN如何在各种各样的问题中实现最先进的结果是无用的。
英文摘要:
In a recently published paper [1], it is shown that deep neural networks (DNNs) with random Gaussian weights preserve the metric structure of the data, with the property that the distance shrinks more when the angle between the two data points is smaller. We agree that the random projection setup considered in [1] preserves distances with a high probability. But as far as we are concerned, the relation between the angle of the data points and the output distances is quite the opposite, i.e., smaller angles result in a weaker distance shrinkage. This leads us to conclude that Theorem 3 and Figure 5 in [1] are not accurate. Hence the usage of random Gaussian weights in DNNs cannot provide an ability of universal classification or treating in-class and out-of-class data separately. Consequently, the behavior of networks consisting of random Gaussian weights only is not useful to explain how DNNs achieve state-of-art results in a large variety of problems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。