Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invariant restricted Boltzmann machine that compactly represents data by its weights and their transformations, which achieves invariance of the feature representation via probabilistic max pooling. In addition, we show that our transformation-invariant feature learning framework can also be extended to other unsupervised learning methods, such as autoencoders or sparse coding. We evaluate our method on several image classification benchmark datasets, such as MNIST variations, CIFAR-10, and STL-10, and show competitive or superior classification performance when compared to the state-of-the-art. Furthermore, our method achieves state-of-the-art performance on phone classification tasks with the TIMIT dataset, which demonstrates wide applicability of our proposed algorithms to other domains.
Robustness to Transformations Across Categories: Is Robustness To Transformations Driven by Invariant Neural Representations?
Massachusetts Institute of Technology（麻省理工学院）；Technical University of Munich（慕尼黑工业大学）；Center for Brains, Minds and Machines（大脑、心智与机器中心）；Fujitsu Research of America（富士通美国研究院）
Out-of-distribution Generalization with Causal Invariant Transformations
University of Chinese Academy of Sciences（中国科学院大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search
University of Kansas（堪萨斯大学）；Southwest University of Finance and Economics（西南财经大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Auckland（奥克兰大学）；Arizona State University（亚利桑那州立大学）；Northeast Normal University（东北师范大学）
