用Benford定律重新思考神经网络
Rethinking Neural Networks With Benford's Law
摘要：
本福德定律（BL）或有效数字定律定义了数据样本中数值的第一位的概率分布。这一定律在许多自然发生的数据集中都可以观察到。它可以被看作是一个给定分布的自然性的度量，并发现其在异常和欺诈检测等领域的应用。在这项工作中，我们解决了以下问题：神经网络参数的分布与网络的泛化能力有关吗？为此，我们首先定义了一个度量指标，MLH（模型焓），它衡量一组数字与本福德定律的接近程度，我们根据经验表明，它是验证准确性的强有力预测。其次，我们使用MLH作为早期停止的验证准确性的替代方案，消除了对验证集的需求。我们提供的实验证据表明，即使事先已知验证集的最佳大小，达到的峰值测试精度也低于根本不使用验证集。最后，我们研究了BL与自由能原理和热力学第一定律的联系，表明MLH是学习系统内部能量的一个组成部分，并且最优化是将总能量最小化以达到平衡的类比。
英文摘要：
Benford's Law (BL) or the Significant Digit Law defines the probability distribution of the first digit of numerical values in a data sample. This Law is observed in many naturally occurring datasets. It can be seen as a measure of naturalness of a given distribution and finds its application in areas like anomaly and fraud detection. In this work, we address the following question: Is the distribution of the Neural Network parameters related to the network's generalization capability? To that end, we first define a metric, MLH (Model Enthalpy), that measures the closeness of a set of numbers to Benford's Law and we show empirically that it is a strong predictor of Validation Accuracy. Second, we use MLH as an alternative to Validation Accuracy for Early Stopping, removing the need for a Validation set. We provide experimental evidence that even if the optimal size of the validation set is known before-hand, the peak test accuracy attained is lower than not using a validation set at all. Finally, we investigate the connection of BL to Free Energy Principle and First Law of Thermodynamics, showing that MLH is a component of the internal energy of the learning system and optimization as an analogy to minimizing the total energy to attain equilibrium.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
