从CNN到基于复小波的平移不变孪生模型
From CNNs to Shift-Invariant Twin Models Based on Complex Wavelets
AI总结：
提出用复值卷积加模运算替代实值卷积加最大池化，构建基于复小波的平移不变孪生模型，在ImageNet和CIFAR-10上提升精度且降低计算与内存开销。
摘要：
我们提出了一种提高卷积神经网络平移不变性和预测精度的新方法。具体而言，我们将第一层组合“实值卷积+最大池化”（RMax）替换为对平移稳定的“复值卷积+模运算”（CMod）。为验证该方法，我们指出当卷积核为带通且定向（类Gabor滤波器）时，CMod与RMax产生可比的输出。因此在此情形下，CMod可被视为RMax的稳定替代方案。为强制该属性，我们约束卷积核采用此类Gabor结构。相应架构被称为数学孪生，因其采用明确定义的数学算子来模拟原始自由训练模型的行为。与基于低通滤波的现有方法相比，我们的方法在ImageNet和CIFAR-10分类任务上取得了更高的准确率。可以说，我们方法强调保留高频细节，有助于在平移不变性与信息保留之间取得更好平衡，从而提升性能。此外，其计算成本和内存占用低于并行工作，使其成为实际应用的有前景方案。
英文摘要：
We propose a novel method to increase shift invariance and prediction accuracy in convolutional neural networks. Specifically, we replace the first-layer combination "real-valued convolutions + max pooling" (RMax) by "complex-valued convolutions + modulus" (CMod), which is stable to translations, or shifts. To justify our approach, we claim that CMod and RMax produce comparable outputs when the convolution kernel is band-pass and oriented (Gabor-like filter). In this context, CMod can therefore be considered as a stable alternative to RMax. To enforce this property, we constrain the convolution kernels to adopt such a Gabor-like structure. The corresponding architecture is called mathematical twin, because it employs a well-defined mathematical operator to mimic the behavior of the original, freely-trained model. Our approach achieves superior accuracy on ImageNet and CIFAR-10 classification tasks, compared to prior methods based on low-pass filtering. Arguably, our approach's emphasis on retaining high-frequency details contributes to a better balance between shift invariance and information preservation, resulting in improved performance. Furthermore, it has a lower computational cost and memory footprint than concurrent work, making it a promising solution for practical implementation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
