We propose a complexity measure of a neural network mapping function based on the diversity of the set of tangent spaces from different inputs. Treating each tangent space as a linear PAC concept we use an entropy-based measure of the bundle of concepts in order to estimate the conceptual capacity of the network. The theoretical maximal capacity of a ReLU network is equivalent to the number of its neurons. In practice however, due to correlations between neuron activities within the network, the actual capacity can be remarkably small, even for very big networks. Empirical evaluations show that this new measure is correlated with the complexity of the mapping function and thus the generalisation capabilities of the corresponding network. It captures the effective, as oppose to the theoretical, complexity of the network function. We also showcase some uses of the proposed measure for analysis and comparison of trained neural network models.
Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning
The University of Tokyo（东京大学）；University of Alberta（阿尔伯塔大学）；Google Research（谷歌研究院）
深度强化学习（RL）研究的进展在很大程度上是由基准任务环境实现的。然而，分析这些环境的性质往往被忽视。特别是，我们仍然没有合适的方法来衡量任务的难度或可解决性，因为每个任务都有根本不同的动作，观察，动态，奖励，并且可以用不同的RL算法来解决。在这项工作中，我们提出了政策信息容量（PIC）-政策参数和情节回报之间的互信息-和政策最优信息容量（POIC）-政策参数和情节最优性之间-作为两个环境不可知，算法不可知的任务难度的定量指标。通过评估我们在玩具环境中的指标以及OpenAI Gym和DeepMind Control Suite的连续控制基准任务，我们实证证明，这些信息理论指标与标准化任务可解决性分数的相关性高于各种替代方案。最后，我们表明，这些指标也可以用于快速和计算效率的优化关键设计参数，如奖励塑造，政策架构，和MDP属性更好的解决方案，由RL算法，而无需运行完整的RL实验。
