像差校正视觉Transformer用于高保真超透镜成像
Aberration Correcting Vision Transformers for High-Fidelity Metalens Imaging
AI总结：
提出一种基于视觉Transformer的超透镜像差校正框架，通过多自适应滤波器引导和空间转置自注意力融合模块，显著提升图像恢复质量并验证了实际制造超透镜的实用性。
摘要：
超透镜是一种新兴的光学系统，具有不可替代的优点，即可以制造为超薄和紧凑的尺寸，在各种应用中展现出巨大潜力。尽管其在小型化方面具有优势，但其实用性受到空间变化像差和畸变的限制，这些像差和畸变显著降低了图像质量。先前的一些工作尝试解决不同类型的像差，但大多数主要针对传统笨重镜头设计，无法有效修复超透镜的严重像差。虽然已有专门针对超透镜的像差校正方法，但它们仍达不到理想的恢复质量。在这项工作中，我们提出了一种新颖的针对超透镜捕获图像的像差校正框架，利用视觉Transformer（ViT）的潜力来恢复具有非均匀像差的超透镜图像。具体而言，我们设计了一个多自适应滤波器引导（MAFG）模块，其中多个维纳滤波器以不同的噪声-细节平衡丰富退化输入图像，并且一个交叉注意力模块根据不同的像差程度对特征进行重新加权。此外，我们引入了一个空间和转置自注意力融合（STAF）模块，该模块聚合来自空间自注意力和转置自注意力模块的特征，以进一步改善像差校正。我们进行了广泛的实验，包括校正像差图像和视频，以及干净的3D重建。所提出的方法显著优于先前的工作。我们进一步制造了一个超透镜，并通过恢复用该制造超透镜捕获的图像来验证我们方法的实用性。代码和预训练模型可在 https://benhenryl.github.io/Metalens-Transformer 获取。
英文摘要：
Metalens is an emerging optical system with an irreplaceable merit in that it can be manufactured in ultra-thin and compact sizes, which shows great promise in various applications. Despite its advantage in miniaturization, its practicality is constrained by spatially varying aberrations and distortions, which significantly degrade the image quality. Several previous arts have attempted to address different types of aberrations, yet most of them are mainly designed for the traditional bulky lens and ineffective to remedy harsh aberrations of the metalens. While there have existed aberration correction methods specifically for metalens, they still fall short of restoration quality. In this work, we propose a novel aberration correction framework for metalens-captured images, harnessing Vision Transformers (ViT) that have the potential to restore metalens images with non-uniform aberrations. Specifically, we devise a Multiple Adaptive Filters Guidance (MAFG), where multiple Wiener filters enrich the degraded input images with various noise-detail balances and a cross-attention module reweights the features considering the different degrees of aberrations. In addition, we introduce a Spatial and Transposed self-Attention Fusion (STAF) module, which aggregates features from spatial self-attention and transposed self-attention modules to further ameliorate aberration correction. We conduct extensive experiments, including correcting aberrated images and videos, and clean 3D reconstruction. The proposed method outperforms the previous arts by a significant margin. We further fabricate a metalens and verify the practicality of our method by restoring the images captured with the manufactured metalens. Code and pre-trained models are available at https://benhenryl.github.io/Metalens-Transformer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
