一种渐进搜索张量环网络的启发式秩选择算法
Heuristic Rank Selection with Progressively Searching Tensor Ring Network
摘要:
最近,张量环网络(TRN)已经应用于深度网络,在压缩率和准确性方面取得了显著的成功。虽然高度相关的TRN的性能,秩选择很少在以前的工作中研究,通常设置为相等的实验。同时,没有任何探试法来选择秩,枚举的方法来找到合适的秩是非常耗时的。有趣的是,我们发现部分排名元素是敏感的,通常聚集在一个狭窄的区域,即感兴趣的区域。因此,基于上述现象,我们提出了一种新型的渐进式遗传算法--渐进搜索张量环网络搜索(PSTRN),该算法具有精确、高效地找到最佳排名的能力。通过进化阶段和渐进阶段,PSTRN可以快速收敛到感兴趣的区域,并获得良好的性能。实验结果表明,与枚举法相比,PSTRN能显著降低求秩的复杂度。此外,我们的方法在MNIST,CIFAR 10/100,UCF 11和HMDB 51等公共基准上进行了验证,达到了最先进的性能。
英文摘要:
Recently, Tensor Ring Networks (TRNs) have been applied in deep networks, achieving remarkable successes in compression ratio and accuracy. Although highly related to the performance of TRNs, rank selection is seldom studied in previous works and usually set to equal in experiments. Meanwhile, there is not any heuristic method to choose the rank, and an enumerating way to find appropriate rank is extremely time-consuming. Interestingly, we discover that part of the rank elements is sensitive and usually aggregate in a narrow region, namely an interest region. Therefore, based on the above phenomenon, we propose a novel progressive genetic algorithm named Progressively Searching Tensor Ring Network Search (PSTRN), which has the ability to find optimal rank precisely and efficiently. Through the evolutionary phase and progressive phase, PSTRN can converge to the interest region quickly and harvest good performance. Experimental results show that PSTRN can significantly reduce the complexity of seeking rank, compared with the enumerating method. Furthermore, our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and HMDB51, achieving the state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。