The recently proposed Multilinear Compressive Learning (MCL) framework combines Multilinear Compressive Sensing and Machine Learning into an end-to-end system that takes into account the multidimensional structure of the signals when designing the sensing and feature synthesis components. The key idea behind MCL is the assumption of the existence of a tensor subspace which can capture the essential features from the signal for the downstream learning task. Thus, the ability to find such a discriminative tensor subspace and optimize the system to project the signals onto that data manifold plays an important role in Multilinear Compressive Learning. In this paper, we propose a novel solution to address both of the aforementioned requirements, i.e., How to find those tensor subspaces in which the signals of interest are highly separable? and How to optimize the sensing and feature synthesis components to transform the original signals to the data manifold found in the first question? In our proposal, the discovery of a high-quality data manifold is conducted by training a nonlinear compressive learning system on the inference task. Its knowledge of the data manifold of interest is then progressively transferred to the MCL components via multi-stage supervised training with the supervisory information encoding how the compressed measurements, the synthesized features, and the predictions should be like. The proposed knowledge transfer algorithm also comes with a semi-supervised adaption that enables compressive learning models to utilize unlabeled data effectively. Extensive experiments demonstrate that the proposed knowledge transfer method can effectively train MCL models to compressively sense and synthesize better features for the learning tasks with improved performances, especially when the complexity of the learning task increases.
Multiview Compressive Coding for 3D Reconstruction
FAIR, Meta AI（元人工智能公司FAIR实验室）
视觉识别的核心目标是从单个图像中理解对象和场景。由于大规模学习和通用表示，2D识别已经取得了巨大的进展。相比之下，3D提出了新的挑战，源于图像中未描绘的遮挡。先前的工作试图通过从多个视图推断或依赖于稀缺的CAD模型和类别特定的先验来克服这些问题，这些先验阻碍了扩展到新的设置。在这项工作中，我们探索单视图三维重建学习的启发，在自监督学习的进步概括表示。我们介绍了一个简单的框架，该框架在单个对象或整个场景的3D点上操作，再加上来自不同RGB-D视频的类别不可知的大规模训练。我们的模型，多视图压缩编码（MCC），学习压缩输入的外观和几何形状，通过查询3D感知解码器来预测3D结构。MCC的通用性和效率使其能够从大规模和多样化的数据源中学习，并对DALL $\cdot$ E 2想象的新对象或用iPhone在野外捕获的新对象具有很强的泛化能力。
Compressive Online Robust Principal Component Analysis with Optical Flow for Video Foreground-Background Separation
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；University of Engineering and Technology, Vietnam National University, Hanoi（越南国立大学河内工程技术大学）
Compressive Hyperspectral Imaging with Side Information
Duke University（杜克大学）
提出了一种盲压缩感知算法，将光谱压缩后的数据进行编码，然后叠加，将三维高光谱数据立方体映射到二维图像上，从而实现了高光谱图像的重建。反演算法通过全局-局部收缩先验从测量值学习字典{ \em in situ}。通过使用RGB图像作为压缩感知系统的边信息，所提出的方法被扩展到从压缩测量和相应的RGB图像的联合数据集学习耦合字典，以提高重建质量。使用硅基液晶调制器构建了一个原型相机。从模拟和真实压缩测量的高光谱数据立方体的实验重建证明了所提出的反演算法的有效性，相机的可行性和边信息的好处。
Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging
Zhejiang University（浙江大学）；Westlake University（西湖大学）；School of Engineering, Westlake University（西湖大学工程学院）；Xidian University（西安电子科技大学）；School of Cyber Engineering, Xidian University（西安电子科技大学网络与工程学院）；Beijing Institute of Technology（北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）
The Power of Triply Complementary Priors for Image Compressive Sensing
Nanyang Technological University（南洋理工大学）；Nokia Bell Labs（诺基亚贝尔实验室）；Institute of High Performance Computing, A*STAR（A*STAR高性能计算研究所）；University of Macau（澳门大学）；University of Electronic Science and Technology of China（电子科技大学）
CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing
School of Computing, Informatics, and Decision Systems Engineering（计算、信息与决策系统工程学院）；Arizona State University（亚利桑那州立大学）
研究了高帧率视频压缩感知（CS）的实时的【形】编解码问题。与以前的作品，使用基于迭代优化的方法进行重建，我们提出了一个非迭代模型，名为“CSDideoNet”。CSDideoNet直接学习CS的逆映射，并在单个前向传播中重建原始输入。为了克服现有CS摄像机的局限性，我们提出了一种多速率CNN和一种合成RNN，以改善重建视频的压缩比（CR）和时空分辨率之间的权衡。实验结果表明，CSDideoNet显着优于国家的最先进的方法。在Titan X GPU上，我们在100倍CR下实现了25 dB的PSNR恢复质量，帧速率为125 fps。由于CSDideoNet的前馈和高数据并发特性，它可以利用GPU加速来实现比传统的基于迭代的方法快三个数量级的速度。我们在www.example.com上共享源代码。
University of Pisa（比萨大学）；Institute of Clinical Physiology, CNR（意大利国家研究委员会临床生理学研究所）；Athinoula A. Martinos Center for Biomedical Imaging（阿瑟努拉·A·马蒂诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Center for Clinical Data Science, MGH（麻省总医院临床数据科学中心）
