一种新的线性时间双层 $\ell_{1,\infty}$ 投影;在自编码器神经网络稀疏化中的应用
A new Linear Time Bi-level $\ell_{1,\infty}$ projection ; Application to the sparsification of auto-encoders neural networks
AI总结:
本文提出一种新的线性时间双层 $\ell_{1,\infty}$ 投影方法,将时间复杂度降至 $\mathcal{O}(nm)$,并验证其比现有最快算法快 $2.5$ 倍,在保持准确率的同时实现最佳稀疏性。
摘要:
$\ell_{1,\infty}$ 范数是一种高效的结构化投影,但遗憾的是,对于 $n\times m$ 矩阵,现有最佳算法的复杂度为 $\mathcal{O}\big(n m \log(n m)\big)$。本文提出一种新的双层投影方法,并证明对于 $n\times m$ 矩阵,其 $\ell_{1,\infty}$ 范数的时间复杂度仅为 $\mathcal{O}\big(n m \big)$。此外,我们提供了一种新的 $\ell_{1,\infty}$ 恒等式,并附有数学证明与实验验证。实验表明,我们的双层 $\ell_{1,\infty}$ 投影比现有最快算法快 $2.5$ 倍,且在分类应用中保持相同准确率的同时提供了最佳稀疏性。
英文摘要:
The $\ell_{1,\infty}$ norm is an efficient-structured projection, but the complexity of the best algorithm is, unfortunately, $\mathcal{O}\big(n m \log(n m)\big)$ for a matrix $n\times m$.\\ In this paper, we propose a new bi-level projection method, for which we show that the time complexity for the $\ell_{1,\infty}$ norm is only $\mathcal{O}\big(n m \big)$ for a matrix $n\times m$. Moreover, we provide a new $\ell_{1,\infty}$ identity with mathematical proof and experimental validation. Experiments show that our bi-level $\ell_{1,\infty}$ projection is $2.5$ times faster than the actual fastest algorithm and provides the best sparsity while keeping the same accuracy in classification applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。