神经网络拓扑结构的博弈分析
Analysing Neural Network Topologies: a Game Theoretic Approach
摘要：
人工神经网络在不同的应用案例中取得了令人印象深刻的成功。选择合适的网络架构是网络成功的关键决策，通常以手动方式完成。作为一种简单的策略，选择大的，大多数是全连接的架构，从而依赖于一个良好的优化策略来找到合适的权重，同时避免过拟合。然而，最终网络的大部分是冗余的。在最好的情况下，网络的大部分变得与后来的推理无关。在最坏的情况下，高度参数化的架构会阻碍适当的优化，并允许轻松创建欺骗网络的adverserial示例。去除不相关的结构部分的第一步是识别这些部分，这需要测量单个组件（如神经元）的贡献。在以前的工作中，基于使用神经元的权重分布作为贡献度量的算法已经取得了一些成功，但没有提供适当的理论理解。因此，在我们的工作中，我们调查博弈论的措施，即Shapley值（SV），以分离相关的人工神经网络的不相关的部分。我们首先为人工神经网络设计一个联盟游戏，其中神经元形成联盟，并且神经元对联盟的平均贡献产生Shapley值。为了衡量Shapley值如何衡量单个神经元的贡献，我们删除了低贡献神经元并测量其对网络性能的影响。在我们的实验中，我们表明，Shapley值优于其他神经元的贡献测量算法。
英文摘要：
Artificial Neural Networks have shown impressive success in very different application cases. Choosing a proper network architecture is a critical decision for a network's success, usually done in a manual manner. As a straightforward strategy, large, mostly fully connected architectures are selected, thereby relying on a good optimization strategy to find proper weights while at the same time avoiding overfitting. However, large parts of the final network are redundant. In the best case, large parts of the network become simply irrelevant for later inferencing. In the worst case, highly parameterized architectures hinder proper optimization and allow the easy creation of adverserial examples fooling the network. A first step in removing irrelevant architectural parts lies in identifying those parts, which requires measuring the contribution of individual components such as neurons. In previous work, heuristics based on using the weight distribution of a neuron as contribution measure have shown some success, but do not provide a proper theoretical understanding. Therefore, in our work we investigate game theoretic measures, namely the Shapley value (SV), in order to separate relevant from irrelevant parts of an artificial neural network. We begin by designing a coalitional game for an artificial neural network, where neurons form coalitions and the average contributions of neurons to coalitions yield to the Shapley value. In order to measure how well the Shapley value measures the contribution of individual neurons, we remove low-contributing neurons and measure its impact on the network performance. In our experiments we show that the Shapley value outperforms other heuristics for measuring the contribution of neurons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
