KeDuSR:通过无核匹配实现真实世界双镜头超分辨率
KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching
AI总结:
本文提出KeDuSR,一种通过无核匹配策略实现真实世界双镜头超分辨率的方法,解决了LR与Ref间分辨率差距导致的匹配失效问题,并构建了DuSR-Real数据集,实验证明其大幅优于现有方法。
摘要:
双镜头超分辨率(SR)是参考(Ref)引导SR的一种实用场景,它利用长焦图像(Ref)来辅助低分辨率广角图像(LR输入)的超分辨率重建。与一般的RefSR不同,双镜头SR中的Ref仅覆盖重叠的视场(FoV)区域。然而,当前的双镜头SR方法很少利用这些特定特征,而是直接在LR输入和Ref之间进行密集匹配。由于LR和Ref之间存在分辨率差距,这种匹配可能会漏掉最佳匹配候选,并破坏重叠FoV区域内的一致结构。与此不同,我们提出首先通过结合全局变形和局部变形,将Ref与LR输入的中心区域(即重叠FoV区域)进行对齐,使对齐后的Ref变得清晰且一致。然后,我们将对齐的Ref和LR中心表述为值-键对,并将LR的角落区域表述为查询。这样,我们提出了一种无核匹配策略,通过在LR角落(查询)和LR中心(键)区域之间进行匹配,相应的对齐Ref(值)可以被变形到目标的角落区域。我们的无核匹配策略避免了LR和Ref之间的分辨率差距,使我们的网络具有更好的泛化能力。此外,我们构建了一个包含(LR, Ref, HR)三元组的DuSR-Real数据集,其中LR和HR已良好对齐。在三个数据集上的实验表明,我们的方法大幅优于次优方法。我们的代码和数据集可在 https://github.com/ZifanCui/KeDuSR 获取。
英文摘要:
Dual-lens super-resolution (SR) is a practical scenario for reference (Ref) based SR by utilizing the telephoto image (Ref) to assist the super-resolution of the low-resolution wide-angle image (LR input). Different from general RefSR, the Ref in dual-lens SR only covers the overlapped field of view (FoV) area. However, current dual-lens SR methods rarely utilize these specific characteristics and directly perform dense matching between the LR input and Ref. Due to the resolution gap between LR and Ref, the matching may miss the best-matched candidate and destroy the consistent structures in the overlapped FoV area. Different from them, we propose to first align the Ref with the center region (namely the overlapped FoV area) of the LR input by combining global warping and local warping to make the aligned Ref be sharp and consistent. Then, we formulate the aligned Ref and LR center as value-key pairs, and the corner region of the LR is formulated as queries. In this way, we propose a kernel-free matching strategy by matching between the LR-corner (query) and LR-center (key) regions, and the corresponding aligned Ref (value) can be warped to the corner region of the target. Our kernel-free matching strategy avoids the resolution gap between LR and Ref, which makes our network have better generalization ability. In addition, we construct a DuSR-Real dataset with (LR, Ref, HR) triples, where the LR and HR are well aligned. Experiments on three datasets demonstrate that our method outperforms the second-best method by a large margin. Our code and dataset are available at https://github.com/ZifanCui/KeDuSR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。