arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1612.01663cs.LG
收藏

基于改进超额风险保证的风险最小化非遗忘随机约简

Efficient Non-oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee

The University of Iowa(爱荷华大学);Hang Seng Management College(恒生管理学院);Nanjing University(南京大学)

Yi Xu, Haiqin Yang, Lijun Zhang, Tianbao Yang

摘要:

在本文中,我们解决高维数据的学习问题。以前,基于不经意随机投影的方法将高维特征投影到随机子空间上,已经在实践中用于解决机器学习中的高维挑战。近年来,各种非遗忘随机化约简方法已经被开发和部署用于解决许多数值问题,例如矩阵乘积近似,低秩矩阵近似等。然而,它们对于机器学习任务的探索较少,例如,分类.更严重的是,理论分析的风险最小化,泛化性能的重要措施,超额风险界尚未建立非遗忘随机约简方法。因此,它仍然是一个开放的问题,使用它们比以前的不经意的随机投影为基础的方法的好处。为了解决这些挑战,我们提出了一个算法框架,采用非遗忘随机约简方法,在机器学习任务中,将原始高维特征投影到一个随机子空间上,该子空间是从具有小矩阵近似误差的数据中导出的。然后,我们推导出所提出的非遗忘随机化减少方法的第一个超额风险界,而不需要对训练数据进行强假设。所建立的超额风险界表明,该方法提供了更好的泛化性能,它也提供了更多的见解不同的随机约简方法。最后,我们进行了广泛的实验,合成和真实世界的基准数据集,其尺寸尺度为$O(10^7)$,证明我们所提出的方法的有效性。

英文摘要:

In this paper, we address learning problems for high dimensional data. Previously, oblivious random projection based approaches that project high dimensional features onto a random subspace have been used in practice for tackling high-dimensionality challenge in machine learning. Recently, various non-oblivious randomized reduction methods have been developed and deployed for solving many numerical problems such as matrix product approximation, low-rank matrix approximation, etc. However, they are less explored for the machine learning tasks, e.g., classification. More seriously, the theoretical analysis of excess risk bounds for risk minimization, an important measure of generalization performance, has not been established for non-oblivious randomized reduction methods. It therefore remains an open problem what is the benefit of using them over previous oblivious random projection based approaches. To tackle these challenges, we propose an algorithmic framework for employing non-oblivious randomized reduction method for general empirical risk minimizing in machine learning tasks, where the original high-dimensional features are projected onto a random subspace that is derived from the data with a small matrix approximation error. We then derive the first excess risk bound for the proposed non-oblivious randomized reduction approach without requiring strong assumptions on the training data. The established excess risk bound exhibits that the proposed approach provides much better generalization performance and it also sheds more insights about different randomized reduction approaches. Finally, we conduct extensive experiments on both synthetic and real-world benchmark datasets, whose dimension scales to $O(10^7)$, to demonstrate the efficacy of our proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑