Important insights towards the explainability of neural networks reside in the characteristics of their decision boundaries. In this work, we borrow tools from the field of adversarial robustness, and propose a new perspective that relates dataset features to the distance of samples to the decision boundary. This enables us to carefully tweak the position of the training samples and measure the induced changes on the boundaries of CNNs trained on large-scale vision datasets. We use this framework to reveal some intriguing properties of CNNs. Specifically, we rigorously confirm that neural networks exhibit a high invariance to non-discriminative features, and show that the decision boundaries of a DNN can only exist as long as the classifier is trained with some features that hold them together. Finally, we show that the construction of the decision boundary is extremely sensitive to small perturbations of the training samples, and that changes in certain directions can lead to sudden invariances in the orthogonal ones. This is precisely the mechanism that adversarial training uses to achieve robustness.
Learning deep neural networks that are generalizable across different domains remains a challenge due to the problem of domain shift. Unsupervised domain adaptation is a promising avenue which transfers knowledge from a source domain to a target domain without using any labels in the target domain. Contemporary techniques focus on extracting domain-invariant features using domain adversarial training. However, these techniques neglect to learn discriminative class boundaries in the latent representation space on a target domain and yield limited adaptation performance. To address this problem,…
Machine learning algorithms using deep architectures have been able to implement increasingly powerful and successful models. However, they also become increasingly more complex, more difficult to comprehend and easier to fool. So far, most methods in the literature investigate the decision of the model for a single given input datum. In this paper, we propose to visualize a part of the decision function of a deep neural network together with a part of the data set in two dimensions with discriminative dimensionality reduction. This enables us to inspect how different properties of the data ar…
Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection
Ampere Software Technology（Ampere软件技术公司）；Institut de mathématiques de Toulouse（图卢兹数学研究所）；Institut de Recherche en Informatique de Toulouse（图卢兹计算机研究所）；Laboratoire Jean Alexandre Dieudonné, Université Côte d’Azur（坎特伯雷大学Dieudonné实验室）
UC Berkeley（加州大学伯克利分校）；ICSI（国际计算机科学研究所）；University of Pennsylvania（宾夕法尼亚大学）；Google Research（谷歌研究院）；MIT（麻省理工学院）
Almost all existing deep learning approaches for semantic segmentation tackle this task as a pixel-wise classification problem. Yet humans understand a scene not in terms of pixels, but by decomposing it into perceptual groups and structures that are the basic building blocks of recognition. This motivates us to propose an end-to-end pixel-wise metric learning approach that mimics this process. In our approach, the optimal visual representation determines the right segmentation within individual images and associates segments with the same semantic classes across images. The core visual learni…
Discriminative Subspace Emersion from learning feature relevances across different populations
University of Birmingham（伯明翰大学）；Imperial College London（伦敦帝国学院）；University of Groningen（格罗宁根大学）；University Hospitals Birmingham NHS Foundation Trust（伯明翰大学医院NHS基金会信托基金）
The Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）
This paper is about authenticating genuine van Gogh paintings from forgeries. The authentication process depends on two key steps: feature extraction and outlier detection. In this paper, a geometric tight frame and some simple statistics of the tight frame coefficients are used to extract features from the paintings. Then a forward stage-wise rank boosting is used to select a small set of features for more accurate classification so that van Gogh paintings are highly concentrated towards some center point while forgeries are spread out as outliers. Numerical results show that our method can a…
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
We are interested in developing a unified machine learning model over many mobile devices for practical learning tasks, where each device only has very few training data. This is a commonly encountered situation in mobile computing scenarios, where data is scarce and distributed while the tasks are distinct. In this paper, we propose a federated few-shot learning (FedFSL) framework to learn a few-shot classification model that can classify unseen data classes with only a few labeled samples. With the federated learning strategy, FedFSL can utilize many data sources while keeping data privacy a…
VAE-Inf: A statistically interpretable generative paradigm for imbalanced classification
Department of Data Science and Artificial Intelligence（数据科学与人工智能系）；The Hong Kong Polytechnic University（香港理工大学）；Department of Applied Mathematics（应用数学系）
Refining Decision Boundaries In Anomaly Detection Using Similarity Search Within the Feature Space
New York University, NYUAD, Division of Science, Computer Science Department（纽约大学）；University of Quebec in Montreal, Computer Science Department, Montreal（魁北克大学蒙特利尔分校）；University of Edinburgh, School of Informatics, Edinburgh（爱丁堡大学）
