基于神经元的神经网络解释牺牲了完备性与可解释性
Neuron-based explanations of neural networks sacrifice completeness and interpretability
AI总结：
该研究针对ImageNet预训练的AlexNet，通过量化指标与用户研究证明，基于神经元的神经网络解释在完备性和可解释性上均不如激活主成分，建议采用主成分等适配分布式表征的基来构建解释方法。
摘要：
高质量的神经网络（NN）解释应具备两项关键属性：完备性确保解释能准确反映网络的功能，可解释性则让人类能够理解这些解释。现有许多方法针对网络内的单个神经元提供解释。在本研究中，我们针对在ImageNet上预训练的AlexNet提供证据表明：与激活主成分相比，基于神经元的解释方法在完备性和可解释性上均有所欠缺。神经元并非AlexNet嵌入的良好基，因为它们未考虑这些表征的分布式特性。通过考察两种完备性的量化指标，并开展用户研究来衡量可解释性，我们证明最重要的主成分能比最重要的神经元提供更完备、更具可解释性的解释。与研究每个神经元不同，仅需考察相对少量的高方差主成分（PC）即可解释大部分激活方差。这些主成分还会对网络功能产生显著影响，且可解释性远高于神经元。我们的发现表明，针对AlexNet这类网络的解释方法应避免将神经元作为嵌入基，而应选择能体现网络内部表征高维与分布式特性的基，例如主成分。交互式演示与代码可访问https://ndey96.github.io/neuron-explanations-sacrifice 获取。
英文摘要：
High quality explanations of neural networks (NNs) should exhibit two key properties. Completeness ensures that they accurately reflect a network's function and interpretability makes them understandable to humans. Many existing methods provide explanations of individual neurons within a network. In this work we provide evidence that for AlexNet pretrained on ImageNet, neuron-based explanation methods sacrifice both completeness and interpretability compared to activation principal components. Neurons are a poor basis for AlexNet embeddings because they don't account for the distributed nature of these representations. By examining two quantitative measures of completeness and conducting a user study to measure interpretability, we show the most important principal components provide more complete and interpretable explanations than the most important neurons. Much of the activation variance may be explained by examining relatively few high-variance PCs, as opposed to studying every neuron. These principal components also strongly affect network function, and are significantly more interpretable than neurons. Our findings suggest that explanation methods for networks like AlexNet should avoid using neurons as a basis for embeddings and instead choose a basis, such as principal components, which accounts for the high dimensional and distributed nature of a network's internal representations. Interactive demo and code available at https://ndey96.github.io/neuron-explanations-sacrifice.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
