压缩感知——简介及其与深度学习的关系
Compressive Sensing - Introduction and Relations to Deep Learning
浏览论文内容
中文总结 AI 辅助
本文介绍压缩感知，阐述其与深度学习的关联，讨论展开式稀疏恢复神经网络的泛化性及梯度下降的隐式正则化，推动两领域交叉发展。
中文摘要 AI 辅助
压缩感知理论表明，稀疏向量（信号）可通过高效算法从少量线性测量值中恢复。这一发现距今已有二十年，它引发了信号处理领域的范式转变，推动了该领域在医学成像、雷达、天文学等实际信号处理应用及理论层面的诸多发展。近来，该领域与深度学习的重要关联催生了进一步进展，例如将展开式神经网络用于稀疏恢复，以及发现常见训练算法（梯度下降的变体）在参数过度设定场景下会偏向稀疏性——即所谓的隐式偏置现象。本文对压缩感知进行介绍，并概述其与深度学习的关联，重点讨论由稀疏恢复算法展开生成的神经网络的泛化能力，以及应用于简化线性神经网络学习的梯度下降的隐式正则化。
英文摘要
Compressive sensing predicts that sparse vectors (signals) can be recovered from a small number of linear measurements via efficient algorithms. This finding, which dates back two decades, has triggered a paradigm shift in signal processing and initiated many developments both in practical signal-processing applications, such as medical imaging, radar, and astronomy, and on the theoretical side. More recently, seminal connections to the field of deep learning have led to further advances in the field, such as the use of unrolled neural networks for sparse recovery and the discovery that common training algorithms (variants of gradient descent) favor sparsity in overparameterized scenarios -- the so-called implicit bias phenomenon. This article gives an introduction to compressive sensing and outlines connections to deep learning. In particular, we will discuss generalization for neural networks generated by unrolling sparse recovery algorithms and implicit regularization for gradient descent applied to learning simplified linear neural networks.
