无混叠卷积网络：通过多项式激活实现分数平移不变性

无混叠ViT：通过线性注意力实现分数平移不变性

Alias-Free ViT: Fractional Shift Invariance via Linear Attention

针对ViT缺乏平移不变性且对图像平移敏感的问题，提出无混叠ViT，通过无混叠下采样、非线性和线性互协方差注意力实现分数平移等变，在保持分类性能的同时提升对抗平移鲁棒性。

Delving Deeper into Anti-aliasing in ConvNets

Aliasing refers to the phenomenon that high frequency signals degenerate into completely different ones after sampling. It arises as a problem in the context of deep learning as downsampling layers are widely adopted in deep architectures to reduce parameters and computation. The standard solution is to apply a low-pass filter (e.g., Gaussian blur) before downsampling. However, it can be suboptimal to apply the same filter across the entire content, as the frequency of feature maps can vary across both spatial locations and feature channels. To tackle this, we propose an adaptive content-aware…

Transfer Learning Based on AdaBoost for Feature Selection from Multiple ConvNet Layer Features

Convolutional Networks (ConvNets) are powerful models that learn hierarchies of visual features, which could also be used to obtain image representations for transfer learning. The basic pipeline for transfer learning is to first train a ConvNet on a large dataset (source task) and then use feed-forward units activation of the trained ConvNet as image representation for smaller datasets (target task). Our key contribution is to demonstrate superior performance of multiple ConvNet layer features over single ConvNet layer features. Combining multiple ConvNet layer features will result in more co…

Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function

Deep convolutional networks are vulnerable to image translation or shift, partly due to common down-sampling layers, e.g., max-pooling and strided convolution. These operations violate the Nyquist sampling rate and cause aliasing. The textbook solution is low-pass filtering (blurring) before down-sampling, which can benefit deep networks as well. Even so, non-linearity units, such as ReLU, often re-introduce the problem, suggesting that blurring alone may not suffice. In this work, first, we analyse deep features with Fourier transform and show that Depth Adaptive Blurring is more effective, a…

Understanding Neural Networks Through Deep Visualization

Recent years have produced great advances in training large, deep neural networks (DNNs), including notable successes in training convolutional neural networks (convnets) to recognize natural images. However, our understanding of how these models work, especially what computations they perform at intermediate layers, has lagged behind. Progress in the field will be further accelerated by the development of better tools for visualizing and interpreting neural nets. We introduce two such tools here. The first is a tool that visualizes the activations produced on each layer of a trained convnet a…

UNet-AF: 一种无别名的UNet用于图像恢复

UNet-AF: Alias-free UNet architectures

本文提出一种无别名UNet架构，通过选择先进的翻译等变层提升图像恢复任务中的等变性，实验显示其在性能和等变性方面优于传统方法。