arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.04687cs.LGeess.SPstat.ML
收藏

全球思维,本地行动:关联DNN概括和节点级SNR

Think Global, Act Local: Relating DNN generalisation and node-level SNR

Paul Norridge

摘要:

良好的DNN泛化背后的原因仍然是一个悬而未决的问题。在本文中,我们探讨这个问题,通过看网络中的节点的信噪比。从信息论原理开始,可以导出DNN节点输出的SNR的表达式。使用这个表达式,我们构建的数字的优点,量化如何以及一个节点的权重优化SNR(或,等价地,信息速率)。应用这些数字的优点,我们给出的例子表明,权重集,促进良好的SNR性能也表现出良好的推广。此外,我们能够确定的质量权重集表现出良好的SNR行为,从而促进良好的推广。这导致了这些结果如何与网络培训和正规化的讨论。最后,我们确定了一些方法,这些意见可以用于培训设计。

英文摘要:

The reasons behind good DNN generalisation remain an open question. In this paper we explore the problem by looking at the Signal-to-Noise Ratio of nodes in the network. Starting from information theory principles, it is possible to derive an expression for the SNR of a DNN node output. Using this expression we construct figures-of-merit that quantify how well the weights of a node optimise SNR (or, equivalently, information rate). Applying these figures-of-merit, we give examples indicating that weight sets that promote good SNR performance also exhibit good generalisation. In addition, we are able to identify the qualities of weight sets that exhibit good SNR behaviour and hence promote good generalisation. This leads to a discussion of how these results relate to network training and regularisation. Finally, we identify some ways that these observations can be used in training design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑