arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.01461cs.CV
收藏

手机上基于相机融合的高效混合变焦

Efficient Hybrid Zoom using Camera Fusion on Mobile Phones

Google(谷歌)

Xiaotong Wu, Wei-Sheng Lai, YiChang Shih, Charles Herrmann, Michael Krainin, Deqing Sun, Chia-Kai Liang

AI总结:

提出一种移动端高效混合变焦超分辨率系统,通过同步捕获广角和长焦图像对并利用机器学习模型将长焦细节转移至广角图像,采用自适应融合处理不匹配等问题,在500ms内生成1200万像素图像且优于现有方法。

摘要:

DSLR相机可以通过改变镜头距离或更换镜头类型来实现多个变焦级别。然而,由于空间限制,这些技术在智能手机设备上无法实现。大多数智能手机制造商采用混合变焦系统:通常在低变焦级别使用广角(W)摄像头,在高变焦级别使用长焦(T)摄像头。为了模拟W和T之间的变焦级别,这些系统会对W的图像进行裁剪和数字上采样,从而导致显著的细节丢失。在本文中,我们提出了一种用于移动设备上混合变焦超分辨率的高效系统,该系统捕获同步的W和T拍摄对,并利用机器学习模型对齐并将细节从T转移到W。我们进一步开发了一种自适应融合方法,该方法考虑了景深不匹配、场景遮挡、光流不确定性和对齐误差。为了最小化域差距,我们设计了一个双手机相机装置,用于捕获真实世界的输入和真实标签以进行监督训练。我们的方法在移动平台上能在500ms内生成一张1200万像素的图像,并且在真实场景的广泛评估中表现优于现有最先进的方法。

英文摘要:

DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems crop and digitally upsample images from W, leading to significant detail loss. In this paper, we propose an efficient system for hybrid zoom super-resolution on mobile devices, which captures a synchronous pair of W and T shots and leverages machine learning models to align and transfer details from T to W. We further develop an adaptive blending method that accounts for depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment errors. To minimize the domain gap, we design a dual-phone camera rig to capture real-world inputs and ground-truths for supervised training. Our method generates a 12-megapixel image in 500ms on a mobile platform and compares favorably against state-of-the-art methods under extensive evaluation on real-world scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑