arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.11972cs.CV
收藏

TransCL:Transformer提供强大而灵活的压缩学习

TransCL: Transformer Makes Strong and Flexible Compressive Learning

Peking University Shenzhen Graduate School(北京大学深圳研究生院);Peng Cheng Laboratory(鹏城实验室)

Chong Mou, Jian Zhang

摘要:

压缩学习(CL)是一种新兴的框架,它集成了通过压缩传感(CS)和机器学习的信号采集,直接对少量的测量推理任务。它可以成为经典图像域方法的一个有前途的替代方案,并且在内存节省和计算效率方面具有很大的优势。然而,以前对CL的尝试不仅限于缺乏灵活性的固定CS比率,而且还限于MNIST/CIFAR类数据集,并且不能扩展到复杂的真实世界高分辨率(HR)数据或视觉任务。在本文中,一种新的基于变换的压缩学习框架的大规模图像与任意CS比率,被称为transCL,提出。具体而言,transCL首先利用可学习的基于块的压缩感知的策略,并提出了一个灵活的线性投影策略,使CL可以在大规模图像上进行有效的逐块方式与任意CS比率。然后,把所有块的CS测量作为一个序列,部署一个纯变压器为基础的骨干执行视觉任务与各种面向任务的头。我们充分的分析表明,TransCL具有很强的抗干扰能力和强大的适应性,以任意CS比率。针对复杂HR数据的大量实验表明,所提出的TransCL在图像分类和语义分割任务中可以实现最先进的性能。特别地,CS比为$10\%$的TransCL可以获得与直接对原始数据进行操作时几乎相同的性能,并且即使CS比极低的$1\%$也仍然可以获得令人满意的性能。我们提出的TransCL的源代码可在\url{https://github.com/MC-E/TransCL/}上获得。

英文摘要:

Compressive learning (CL) is an emerging framework that integrates signal acquisition via compressed sensing (CS) and machine learning for inference tasks directly on a small number of measurements. It can be a promising alternative to classical image-domain methods and enjoys great advantages in memory saving and computational efficiency. However, previous attempts on CL are not only limited to a fixed CS ratio, which lacks flexibility, but also limited to MNIST/CIFAR-like datasets and do not scale to complex real-world high-resolution (HR) data or vision tasks. In this paper, a novel transformer-based compressive learning framework on large-scale images with arbitrary CS ratios, dubbed TransCL, is proposed. Specifically, TransCL first utilizes the strategy of learnable block-based compressed sensing and proposes a flexible linear projection strategy to enable CL to be performed on large-scale images in an efficient block-by-block manner with arbitrary CS ratios. Then, regarding CS measurements from all blocks as a sequence, a pure transformer-based backbone is deployed to perform vision tasks with various task-oriented heads. Our sufficient analysis presents that TransCL exhibits strong resistance to interference and robust adaptability to arbitrary CS ratios. Extensive experiments for complex HR data demonstrate that the proposed TransCL can achieve state-of-the-art performance in image classification and semantic segmentation tasks. In particular, TransCL with a CS ratio of $10\%$ can obtain almost the same performance as when operating directly on the original data and can still obtain satisfying performance even with an extremely low CS ratio of $1\%$. The source codes of our proposed TransCL is available at \url{https://github.com/MC-E/TransCL/}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑