In this paper, we address learning problems for high dimensional data. Previously, oblivious random projection based approaches that project high dimensional features onto a random subspace have been used in practice for tackling high-dimensionality challenge in machine learning. Recently, various non-oblivious randomized reduction methods have been developed and deployed for solving many numerical problems such as matrix product approximation, low-rank matrix approximation, etc. However, they are less explored for the machine learning tasks, e.g., classification. More seriously, the theoretical analysis of excess risk bounds for risk minimization, an important measure of generalization performance, has not been established for non-oblivious randomized reduction methods. It therefore remains an open problem what is the benefit of using them over previous oblivious random projection based approaches. To tackle these challenges, we propose an algorithmic framework for employing non-oblivious randomized reduction method for general empirical risk minimizing in machine learning tasks, where the original high-dimensional features are projected onto a random subspace that is derived from the data with a small matrix approximation error. We then derive the first excess risk bound for the proposed non-oblivious randomized reduction approach without requiring strong assumptions on the training data. The established excess risk bound exhibits that the proposed approach provides much better generalization performance and it also sheds more insights about different randomized reduction approaches. Finally, we conduct extensive experiments on both synthetic and real-world benchmark datasets, whose dimension scales to $O(10^7)$, to demonstrate the efficacy of our proposed approach.
How robust is randomized blind deconvolution via nuclear norm minimization against adversarial noise?
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Munich Data Science Institute(慕尼黑数据科学研究所);KU Eichstätt-Ingolstadt(艾希施泰特-英戈尔施塔特天主教大学)
Randomized Smoothing SVRG for Large-scale Nonsmooth Convex Optimization
National University of Singapore(新加坡国立大学);Dept. of Industrial Systems Engineering and Management(工业系统工程与管理系(此处按要求粒度,该系属于新加坡国立大学,机构级别为大学,原部门为系,非独立机构,按要求只提取到大学/学院/公司/研究院级别,所以主要机构为新加坡国立大学,部门系不提取)。修正:根据规则,机构粒度提取到大学/学院/公司/研究院级别即可,不要细拆到系。所以这里的机构是National University of Singapore(新加坡国立大学),而Dept. of Industrial Systems Engineering and Management是系,属于新加坡国立大学内部的系,不应提取。因此正确输出应为{)