We initiate the rigorous study of classification in semimetric spaces, which are point sets with a distance function that is non-negative and symmetric, but need not satisfy the triangle inequality. For metric spaces, the doubling dimension essentially characterizes both the runtime and sample complexity of classification algorithms --- yet we show that this is not the case for semimetrics. Instead, we define the {\em density dimension} and discover that it plays a central role in the statistical and algorithmic feasibility of learning in semimetric spaces. We present nearly optimal sample compression algorithms and use these to obtain generalization guarantees, including fast rates. The latter hold for general sample compression schemes and may be of independent interest.
We propose a new embedding method for a single vector and for a pair of vectors. This embedding method enables: a) efficient classification and regression of functions of single vectors; b) efficient approximation of distance functions; and c) non-Euclidean, semimetric learning. To the best of our knowledge, this is the first work that enables learning any general, non-Euclidean, semimetrics. That is, our method is a universal semimetric learning and approximation method that can approximate any distance function with as high accuracy as needed with or without semimetric constraints. The proje…
International Institute of Information Technology Bangalore（班加罗尔国际信息技术研究所）
Standard metrics of performance of classifiers, such as accuracy and sensitivity, do not reveal the trust or confidence in the predicted labels of data. While other metrics such as the computed probability of a label or the signed distance from a hyperplane can act as a trust measure, these are subjected to heuristic thresholds. This paper presents a convex optimization-based supervised classifier that sequentially partitions a dataset into several ellipsoids, where each ellipsoid contains nearly all points of the same label. By stating classification rules based on this partitioning, Bayes' f…
University of California, San Diego（加州大学圣地亚哥分校）；Halicioğlu Data Science Institute（哈利西奥卢数据科学研究所）
Discriminating between distributions is an important problem in a number of scientific fields. This motivated the introduction of Linear Optimal Transportation (LOT), which embeds the space of distributions into an $L^2$-space. The transform is defined by computing the optimal transport of each distribution to a fixed reference distribution, and has a number of benefits when it comes to speed of computation and to determining classification boundaries. In this paper, we characterize a number of settings in which LOT embeds families of distributions into a space in which they are linearly separ…
We describe a novel spiking neural network (SNN) for automated, real-time handwritten digit classification and its implementation on a GP-GPU platform. Information processing within the network, from feature extraction to classification is implemented by mimicking the basic aspects of neuronal spike initiation and propagation in the brain. The feature extraction layer of the SNN uses fixed synaptic weight maps to extract the key features of the image and the classifier layer uses the recently developed NormAD approximate gradient descent based supervised learning algorithm for spiking neural n…
National Tsing Hua University（国立清华大学）；Konkuk University（建国大学）；University of Washington（华盛顿大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；University of California, Berkeley（加州大学伯克利分校）
The modeling of jet substructure significantly differs between Parton Shower Monte Carlo (PSMC) programs. Despite this, we observe that machine learning classifiers trained on different PSMCs learn nearly the same function. This means that when these classifiers are applied to the same PSMC for testing, they result in nearly the same performance. This classifier universality indicates that a machine learning model trained on one simulation and tested on another simulation (or data) will likely be optimal. Our observations are based on detailed studies of shallow and deep neural networks applie…
Ariel University（阿里埃勒大学）；Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Tel-Aviv University（特拉维夫大学）
We present the first sample compression algorithm for nearest neighbors with non-trivial performance guarantees. We complement these guarantees by demonstrating almost matching hardness lower bounds, which show that our bound is nearly optimal. Our result yields new insight into margin-based nearest neighbor classification in metric spaces and allows us to significantly sharpen and simplify existing bounds. Some encouraging empirical results are also presented.
JMA: a General Algorithm to Craft Nearly Optimal Targeted Adversarial Example
Department of Information Engineering and Mathematics, University of Siena（信息工程与数学系，锡耶纳大学）；Department of Electrical and Electronic Engineering, University of Cagliari（电气与电子工程系，卡利亚里大学）
Bandit-Feedback Online Multiclass Classification: Variants and Tradeoffs
The Henry and Marilyn Taub Faculty of Computer Science, Technion（以色列理工学院亨利与玛丽莲·陶布计算机科学学院）；Faculty of Mathematics, Technion（以色列理工学院数学学院）；Department of Computer Science, Purdue University（普渡大学计算机科学系）；Faculty of Data and Decision Sciences, Technion（以色列理工学院数据与决策科学学院）；Google Research（谷歌研究院）
Traditional classification algorithms assume that training and test data come from similar distributions. This assumption is violated in adversarial settings, where malicious actors modify instances to evade detection. A number of custom methods have been developed for both adversarial evasion attacks and robust learning. We propose the first systematic and general-purpose retraining framework which can: a) boost robustness of an \emph{arbitrary} learning algorithm, in the face of b) a broader class of adversarial models than any prior methods. We show that, under natural conditions, the retra…
Beijing Institute of Technology（北京理工大学）；Huazhong University of Science and Technology（华中科技大学）
Node classification in structural networks has been proven to be useful in many real world applications. With the development of network embedding, the performance of node classification has been greatly improved. However, nearly all the existing network embedding based methods are hard to capture the actual category features of a node because of the linearly inseparable problem in low-dimensional space; meanwhile they cannot incorporate simultaneously network structure information and node label information into network embedding. To address the above problems, in this paper, we propose a nov…
OWL University of Applied Sciences and Arts（奥斯特法利亚应用科学大学）；University Hospital Schleswig-Holstein (UKSH)（石勒苏益格-荷尔斯泰因大学医院）；AiNT GmbH（AiNT有限公司）
There is a pressing market demand to minimize the test time of Prompt Gamma Neutron Activation Analysis (PGNAA) spectra measurement machine, so that it could function as an instant material analyzer, e.g. to classify waste samples instantaneously and determine the best recycling method based on the detected compositions of the testing sample. This article introduces a new development of the deep learning classification and contrive to reduce the test time for PGNAA machine. We propose both Random Sampling Methods and Class Activation Map (CAM) to generate "downsized" samples and train the CNN …
Columbia University（哥伦比亚大学）；Georgia Institute of Technology（佐治亚理工学院）
The support vector machine (SVM) is a well-established classification method whose name refers to the particular training examples, called support vectors, that determine the maximum margin separating hyperplane. The SVM classifier is known to enjoy good generalization properties when the number of support vectors is small compared to the number of training examples. However, recent research has shown that in sufficiently high-dimensional linear classification problems, the SVM can generalize well despite a proliferation of support vectors where all training examples are support vectors. In th…
