Much of Bayesian inference centers around the design of estimators for inverse problems which are optimal assuming the data comes from a known prior. But what do these optimality guarantees mean if the prior is unknown? In recent years, algorithm unrolling has emerged as deep learning's answer to this age-old question: design a neural network whose layers can in principle simulate iterations of inference algorithms and train on data generated by the unknown prior. Despite its empirical success, however, it has remained unclear whether this method can provably recover the performance of its optimal, prior-aware counterparts.
In this work, we prove the first rigorous learning guarantees for neural networks based on unrolling approximate message passing (AMP). For compressed sensing, we prove that when trained on data drawn from a product prior, the layers of the network approximately converge to the same denoisers used in Bayes AMP. We also provide extensive numerical experiments for compressed sensing and rank-one matrix estimation demonstrating the advantages of our unrolled architecture - in addition to being able to obliviously adapt to general priors, it exhibits improvements over Bayes AMP in more general settings of low dimensions, non-Gaussian designs, and non-product priors.
Step-unrolled Denoising Autoencoders for Text Generation
DeepMind（深度思维）
在本文中，我们提出了一种新的文本生成模型，步进展开去噪自编码器（SUNDAE），它不依赖于自回归模型。与去噪扩散技术类似，SUNDAE重复应用于一系列令牌，从随机输入开始，每次改进，直到收敛。我们提出了一个简单的新的改进算子，收敛在更少的迭代比扩散方法，同时定性产生更好的样本自然语言数据集。SUNDAE在WMT'14英语到德语翻译任务上取得了最先进的结果（在非自回归方法中），在Colossal Cleaned Common Crawl数据集和GitHub的Python代码数据集上取得了良好的无条件语言建模定性结果。SUNDAE的非自回归性质通过在模板中填充任意空白模式，打开了从左到右提示生成之外的可能性。
Self-supervised Deep Unrolled Reconstruction Using Regularization by Denoising
State University of New York at Buffalo（纽约州立大学布法罗分校）；Cleveland Clinic（克利夫兰诊所）；Paul C. Lauterbur Research Center for Biomedical Imaging, SIAT, CAS（中国科学院深圳先进技术研究院保罗·C·劳特伯生物医学成像研究中心）
Attention-Only Transformers via Unrolled Subspace Denoising
Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor（电气工程与计算机科学系，密歇根大学，安阿伯分校）；Department of Electrical Engineering and Computer Science, University of California, Berkeley（电气工程与计算机科学系，加州大学伯克利分校）；Institute of Data Science & School of Computing and Data Science, University of Hong Kong（数据科学研究所及计算与数据科学学院，香港大学）
Posterior Sampling with Denoising Oracles via Tilted Transport
Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Center for Data Science, New York University（纽约大学数据科学中心）；Flatiron Institute（弗莱堡研究所）
Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie
University of Oxford（牛津大学）；Institut Universitaire de France（法国大学研究院）；Heriot-Watt University（赫瑞瓦特大学）；Maxwell Institute for Mathematical Sciences（麦克斯韦数学科学研究所）
本文针对贝叶斯成像中 Plug & Play (PnP) 先验方法缺乏收敛证明的问题，提出了两种可证明收敛的算法 PnP-ULA 和 PnP-SGD，并建立了详细的收敛保证，适用于图像去模糊、修复和去噪等任务。
A Master Class on Reproducibility: A Student Hackathon on Advanced MRI Reconstruction Methods
Technical University of Munich (TUM)（技术大学慕尼黑）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（生物医学成像机器学习研究所，海德堡慕尼黑）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；School of Medicine and Health, TUM University Hospital Rechts der Isar（医学与健康学院，技术大学慕尼黑医院Rechts der Isar）；Department of Computing, Imperial College London（计算学院，伦敦帝国理工学院）
University of California, Berkeley（加州大学伯克利分校）；Massachusetts Institute of Technology（麻省理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Microsoft Research（微软研究院）；Carnegie Mellon University（卡内基梅隆大学）；Duke University（杜克大学）
Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
RIKEN AIP & The Institute of Statistical Mathematics（日本理化学研究所AIP及统计数学研究所）；University of Sydney（悉尼大学）；Agency for Science, Technology and Research & The Institute of Statistical Mathematics（科技研究局及统计数学研究所）；The University of Tokyo（东京大学）；Nanyang Technological University（南洋理工大学）；The Institute of Statistical Mathematics（统计数学研究所）
Assessment of Data Consistency through Cascades of Independently Recurrent Inference Machines for fast and robust accelerated MRI reconstruction
Amsterdam UMC（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；MS Center Amsterdam（阿姆斯特丹多发性硬化中心）；Amsterdam Neuroscience（阿姆斯特丹神经科学研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Leiden University（莱顿大学）；Delft University of Technology（代尔夫特理工大学）
