Learning from Incomplete Features by Simultaneous Training of Neural Networks and Sparse Coding
RIKEN（理化学研究所）；Instituto Argentino de Radioastronomía, CONICET-CCT La Plata / CIC-PBA / UNLP（阿根廷射电天文研究所，CONICET-拉普拉塔科技中心 / 布宜诺斯艾利斯省科学调查委员会 / 拉普拉塔国立大学）；Southeast University（东南大学）；University of Vic - Central University of Catalonia（维克大学-加泰罗尼亚中央大学）
Cesar F. Caiafa, Ziyao Wang, Jordi Solé-Casals, Qibin Zhao
备注：
11 pages, 7 figures, paper accepted for presentation at L2ID Workshop at CVPR 2021 (19-25 June, 2021)
In this paper, the problem of training a classifier on a dataset with incomplete features is addressed. We assume that different subsets of features (random or structured) are available at each data instance. This situation typically occurs in the applications when not all the features are collected for every data sample. A new supervised learning method is developed to train a general classifier, such as a logistic regression or a deep neural network, using only a subset of features per sample, while assuming sparse representations of data vectors on an unknown dictionary. Sufficient conditions are identified, such that, if it is possible to train a classifier on incomplete observations so that their reconstructions are well separated by a hyperplane, then the same classifier also correctly separates the original (unobserved) data samples. Extensive simulation results on synthetic and well-known datasets are presented that validate our theoretical findings and demonstrate the effectiveness of the proposed method compared to traditional data imputation approaches and one state-of-the-art algorithm.
Technical University of Munich（慕尼黑工业大学）；Ludwig-Maximilians University（慕尼黑大学）
Large-scale population-based studies in medicine are a key resource towards better diagnosis, monitoring, and treatment of diseases. They also serve as enablers of clinical decision support systems, in particular Computer Aided Diagnosis (CADx) using machine learning (ML). Numerous ML approaches for CADx have been proposed in literature. However, these approaches assume full data availability, which is not always feasible in clinical data. To account for missing data, incomplete data samples are either removed or imputed, which could lead to data bias and may negatively affect classification p…
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Sparse coding approximates the data sample as a sparse linear combination of some basic codewords and uses the sparse codes as new presentations. In this paper, we investigate learning discriminative sparse codes by sparse coding in a semi-supervised manner, where only a few training samples are labeled. By using the manifold structure spanned by the data set of both labeled and unlabeled samples and the constraints provided by the labels of the labeled samples, we learn the variable class labels for all the samples. Furthermore, to improve the discriminative ability of the learned sparse code…
Inspired by recent work on convex formulations of clustering (Lashkari & Golland, 2008; Nowozin & Bakir, 2008) we investigate a new formulation of the Sparse Coding Problem (Olshausen & Field, 1997). In sparse coding we attempt to simultaneously represent a sequence of data-vectors sparsely (i.e. sparse approximation (Tropp et al., 2006)) in terms of a 'code' defined by a set of basis elements, while also finding a code that enables such an approximation. As existing alternating optimization procedures for sparse coding are theoretically prone to severe local minima problems, we propose a conv…
In this paper, we consider the problem of multi-view clustering on incomplete views. Compared with complete multi-view clustering, the view-missing problem increases the difficulty of learning common representations from different views. To address the challenge, we propose a novel incomplete multi-view clustering framework, which incorporates cross-view relation transfer and multi-view fusion learning. Specifically, based on the consistency existing in multi-view data, we devise a cross-view relation transfer-based completion module, which transfers known similar inter-instance relationships …
Joint SDU-NTU Centre for Artificial Intelligence Research, Shandong University（山东大学-南洋理工大学人工智能联合研究中心,山东大学）；College of Computer and Information Sciences, Southwest University（西南大学计算机与信息科学学院）；School of Software, Shandong University（山东大学软件学院）；CEMSE, King Abdullah University of Science and Technology（计算机、工程与数学科学学院,沙特阿拉伯国王阿卜杜拉科技大学）；Department of Computer Science, George Mason University（乔治梅森大学计算机科学系）
Multi-view clustering aims at exploiting information from multiple heterogeneous views to promote clustering. Most previous works search for only one optimal clustering based on the predefined clustering criterion, but devising such a criterion that captures what users need is difficult. Due to the multiplicity of multi-view data, we can have meaningful alternative clusterings. In addition, the incomplete multi-view data problem is ubiquitous in real world but has not been studied for multiple clusterings. To address these issues, we introduce a deep incomplete multi-view multiple clusterings …
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Tsinghua University（清华大学）；Yahoo! Research（雅虎研究院）；Microsoft（微软）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Multi-label learning deals with the classification problems where each instance can be assigned with multiple labels simultaneously. Conventional multi-label learning approaches mainly focus on exploiting label correlations. It is usually assumed, explicitly or implicitly, that the label sets for training instances are fully labeled without any missing labels. However, in many real-world multi-label datasets, the label assignments for training instances can be incomplete. Some ground-truth labels can be missed by the labeler from the label set. This problem is especially typical when the numbe…
Center for Information in BioMedicine（生物医学信息中心）；China Center for Applied Mathematics（中国应用数学中心）
Recently sparse coding have been highly successful in image classification mainly due to its capability of incorporating the sparsity of image representation. In this paper, we propose an improved sparse coding model based on linear spatial pyramid matching(SPM) and Scale Invariant Feature Transform (SIFT ) descriptors. The novelty is the simultaneous non-convex and non-negative characters added to the sparse coding model. Our numerical experiments show that the improved approach using non-convex and non-negative sparse coding is superior than the original ScSPM[1] on several typical databases…
A sparse coding approach to inverse problems with application to microwave tomography
Instituto Argentino de Radioastronomía（阿根廷射电天文研究所）；CONICET CCT La Plata（阿根廷国家科学与技术研究委员会拉普拉塔科学技术中心）；CIC-PBA（布宜诺斯艾利斯省科学研究委员会）；UNLP（拉普拉塔国立大学）；Instituto de Física de Líquidos y Sistemas Biológicos（液体与生物系统物理研究所）
With the continued innovations of deep neural networks, spiking neural networks (SNNs) that more closely resemble biological brain synapses have attracted attention owing to their low power consumption.However, for continuous data values, they must employ a coding process to convert the values to spike trains.Thus, they have not yet exceeded the performance of artificial neural networks (ANNs), which handle such values directly.To this end, we combine an ANN and an SNN to build versatile hybrid neural networks (HNNs) that improve the concerned performance.To qualify this performance, MNIST and…
National University of Defense Technology（国防科技大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；Hainan University（海南大学）；Hainan blockchain technology engineering research center（海南区块链技术工程研究中心）
With the development of various applications, such as social networks and knowledge graphs, graph data has been ubiquitous in the real world. Unfortunately, graphs usually suffer from being absent due to privacy-protecting policies or copyright restrictions during data collection. The absence of graph data can be roughly categorized into attribute-incomplete and attribute-missing circumstances. Specifically, attribute-incomplete indicates that a part of the attribute vectors of all nodes are incomplete, while attribute-missing indicates that the whole attribute vectors of partial nodes are mis…
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The University of Queensland（昆士兰大学）
Clustering with incomplete views is a challenge in multi-view clustering. In this paper, we provide a novel and simple method to address this issue. Specifically, the proposed method simultaneously exploits the local information of each view and the complementary information among views to learn the common latent representation for all samples, which can greatly improve the compactness and discriminability of the obtained representation. Compared with the conventional graph embedding methods, the proposed method does not introduce any extra regularization term and corresponding penalty paramet…
Georgia Institute of Technology（佐治亚理工学院）；DeepMind Technologies（DeepMind科技公司）；Princeton University（普林斯顿大学）；The City College of New York（纽约城市学院）；University of Virginia（弗吉尼亚大学）
In this paper, we propose a new unsupervised feature learning framework, namely Deep Sparse Coding (DeepSC), that extends sparse coding to a multi-layer architecture for visual object recognition tasks. The main innovation of the framework is that it connects the sparse-encoders from different layers by a sparse-to-dense module. The sparse-to-dense module is a composition of a local spatial pooling step and a low-dimensional embedding process, which takes advantage of the spatial smoothness information in the image. As a result, the new method is able to learn several levels of sparse represen…
MRC Brain Network Dynamics Unit, University of Oxford（牛津大学医学研究委员会脑网络动力学研究中心）；University of Oxford（牛津大学）
The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning. However, it requires sequential backward updates and non-local computations, which make it challenging to parallelize at scale and is unlike how learning works in the brain. Neuroscience-inspired learning algorithms, however, such as \emph{predictive coding}, which utilize local learning, have the potential to overcome these limitations and advance beyond current deep learning technologies. While predictive coding originated in theoretical neuroscience as a model of…
