Maxpooling的理论表达
The Theoretical Expressiveness of Maxpooling
摘要：
自从深度神经网络成为最先进的图像分类器以来的十年里，有一种减少使用最大池的趋势：最大池是图像中最大的邻近像素的函数。由于最大池在早期的图像分类器中表现突出，我们希望了解这种趋势，以及它是否合理。我们开发了一个理论框架，分析基于ReLU的最大池的近似，并证明了使用ReLU激活无法有效复制最大池的意义。我们分析了一类最佳近似的错误，并发现，虽然错误可以指数小的内核大小，这样做需要一个指数复杂的近似。我们的工作提供了一个理论基础，了解远离最大池在新的架构的趋势。我们的结论是，最大池和最佳近似值之间的差异的主要原因，池内的最大值和其他值之间普遍存在很大的差异，可以克服与其他架构的决定，或者是不普遍的自然图像。
英文摘要：
Over the decade since deep neural networks became state of the art image classifiers there has been a tendency towards less use of max pooling: the function that takes the largest of nearby pixels in an image. Since max pooling featured prominently in earlier generations of image classifiers, we wish to understand this trend, and whether it is justified. We develop a theoretical framework analyzing ReLU based approximations to max pooling, and prove a sense in which max pooling cannot be efficiently replicated using ReLU activations. We analyze the error of a class of optimal approximations, and find that whilst the error can be made exponentially small in the kernel size, doing so requires an exponentially complex approximation. Our work gives a theoretical basis for understanding the trend away from max pooling in newer architectures. We conclude that the main cause of a difference between max pooling and an optimal approximation, a prevalent large difference between the max and other values within pools, can be overcome with other architectural decisions, or is not prevalent in natural images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
