激发丢失：鼓励深层神经网络的可塑性
Excitation Dropout: Encouraging Plasticity in Deep Neural Networks
摘要：
我们提出了一种用于深度网络的引导式dropout正则化器，其基于网络预测的证据，该网络预测被定义为特定路径中神经元的激发。在这项工作中，我们利用每个神经元的证据来确定dropout的概率，而不是像标准dropout那样随机均匀地丢弃神经元。本质上，我们以更高的概率丢弃那些在训练时对决策做出更多贡献的神经元。这种方法惩罚与模型预测最相关的高显着性神经元，即那些具有更强证据的神经元。通过丢弃这种高显着性神经元，网络被迫学习替代路径，以保持损失最小化，从而导致类似可塑性的行为，这也是人类大脑的特征。我们表现出更好的泛化能力，提高网络神经元的利用率，以及更高的弹性网络压缩使用几个指标超过四个图像/视频识别基准。
英文摘要：
We propose a guided dropout regularizer for deep networks based on the evidence of a network prediction defined as the firing of neurons in specific paths. In this work, we utilize the evidence at each neuron to determine the probability of dropout, rather than dropping out neurons uniformly at random as in standard dropout. In essence, we dropout with higher probability those neurons which contribute more to decision making at training time. This approach penalizes high saliency neurons that are most relevant for model prediction, i.e. those having stronger evidence. By dropping such high-saliency neurons, the network is forced to learn alternative paths in order to maintain loss minimization, resulting in a plasticity-like behavior, a characteristic of human brains too. We demonstrate better generalization ability, an increased utilization of network neurons, and a higher resilience to network compression using several metrics over four image/video recognition benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
