尽可能大：通过傅里叶域中的神经隐式函数学习无限大滤波器
As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domain
AI总结：
该研究提出通过傅里叶域神经隐式函数学习大尺寸滤波器的即插即用模块，在不增加可学习参数的前提下实现大卷积核的高效训练，分析发现学习到的滤波器实际局部性好、尺寸较小，为构建高效高性能模型提供了新思路。
摘要：
近年来，用于图像分类的神经网络研究呈现出明显的空间上下文增大趋势。无论是通过大卷积核还是自注意力实现，模型的扩展性都会随着空间上下文的增加而变差，因此模型精度的提升往往伴随着巨大的成本。本文提出了一个用于研究卷积神经网络有效滤波器尺寸的模块。要开展这类研究，需要解决若干挑战：1）我们需要一种有效的方法来训练带有大滤波器（尺寸可能与输入数据一样大）的模型，同时不增加可学习参数的数量；2）所采用的卷积操作应当是一个即插即用模块，能够替换CNN中的传统卷积，并且可以在现有框架中高效实现；3）滤波器尺寸的研究必须与网络宽度或可学习参数数量等其他因素解耦；4）卷积操作本身的成本必须保持在可控范围内，也就是说我们不能简单地增大卷积核的尺寸。为应对这些挑战，我们提出将滤波器权重的频率表示作为神经隐式函数进行学习，从而利用频域中卷积更好的可扩展性。此外，由于所提出的神经隐式函数的实现方式，即使是大尺寸且具有强表达能力的空间滤波器，也仅需少量可学习权重即可参数化。我们的分析表明，尽管所提出的网络能够学习非常大的卷积核，但在从频域转换到空间域时，学习到的滤波器在实际中具有良好的局部性且尺寸相对较小。我们预计，对单独优化的滤波器尺寸的分析将在未来催生更高效且有效的模型。相关代码可参见https://github.com/GeJulia/NIFF。
英文摘要：
Recent work in neural networks for image classification has seen a strong tendency towards increasing the spatial context. Whether achieved through large convolution kernels or self-attention, models scale poorly with the increased spatial context, such that the improved model accuracy often comes at significant costs. In this paper, we propose a module for studying the effective filter size of convolutional neural networks. To facilitate such a study, several challenges need to be addressed: 1) we need an effective means to train models with large filters (potentially as large as the input data) without increasing the number of learnable parameters 2) the employed convolution operation should be a plug-and-play module that can replace conventional convolutions in a CNN and allow for an efficient implementation in current frameworks 3) the study of filter sizes has to be decoupled from other aspects such as the network width or the number of learnable parameters 4) the cost of the convolution operation itself has to remain manageable i.e. we cannot naively increase the size of the convolution kernel. To address these challenges, we propose to learn the frequency representations of filter weights as neural implicit functions, such that the better scalability of the convolution in the frequency domain can be leveraged. Additionally, due to the implementation of the proposed neural implicit function, even large and expressive spatial filters can be parameterized by only a few learnable weights. Our analysis shows that, although the proposed networks could learn very large convolution kernels, the learned filters are well localized and relatively small in practice when transformed from the frequency to the spatial domain. We anticipate that our analysis of individually optimized filter sizes will allow for more efficient, yet effective, models in the future. https://github.com/GeJulia/NIFF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
