In this paper, we propose a panorama stitching algorithm based on asymmetric bidirectional optical flow. This algorithm expects multiple photos captured by fisheye lens cameras as input, and then, through the proposed algorithm, these photos can be merged into a high-quality 360-degree spherical panoramic image. For photos taken from a distant perspective, the parallax among them is relatively small, and the obtained panoramic image can be nearly seamless and undistorted. For photos taken from a close perspective or with a relatively large parallax, a seamless though partially distorted panoramic image can also be obtained. Besides, with the help of Graphics Processing Unit (GPU), this algorithm can complete the whole stitching process at a very fast speed: typically, it only takes less than 30s to obtain a panoramic image of 9000-by-4000 pixels, which means our panorama stitching algorithm is of high value in many real-time applications. Our code is available at https://github.com/MungoMeng/Panorama-OpticalFlow.
National Yang Ming Chiao Tung University（国立阳明交通大学）；ByteDance（字节跳动）
We propose a novel approach to compute high-resolution (2048x1024 and higher) depths for panoramas that is significantly faster and qualitatively and qualitatively more accurate than the current state-of-the-art method (360MonoDepth). As traditional neural network-based methods have limitations in the output image sizes (up to 1024x512) due to GPU memory constraints, both 360MonoDepth and our method rely on stitching multiple perspective disparity or depth images to come out a unified panoramic depth map. However, to achieve globally consistent stitching, 360MonoDepth relied on solving extensi…
Johns Hopkins University（约翰斯·霍普金斯大学）；Technical University of Munich（慕尼黑工业大学）
Stitching images acquired under perspective projective geometry is a relevant topic in computer vision with multiple applications ranging from smartphone panoramas to the construction of digital maps. Image stitching is an equally prominent challenge in medical imaging, where the limited field-of-view captured by single images prohibits holistic analysis of patient anatomy. The barrier that prevents straight-forward mosaicing of 2D images is depth mismatch due to parallax. In this work, we leverage the Fourier slice theorem to aggregate information from multiple transmission images in parallax…
Panorama creation is one of the most widely deployed techniques in computer vision. In addition to industry applications such as Google Street View, it is also used by millions of consumers in smartphones and other cameras. Traditionally, the problem is decomposed into three phases: registration, which picks a single transformation of each source image to align it to the other inputs, seam finding, which selects a source image for each pixel in the final result, and blending, which fixes minor visual artifacts. Here, we observe that the use of a single registration often leads to errors, espec…
StabStitch++: Unsupervised Online Video Stitching with Spatiotemporal Bidirectional Warps
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing（教育部视觉智能+X国际合作联合实验室）；School of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Media Engineering, Communication University of Zhejiang（浙江传媒学院媒体工程学院）；School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）
The naturalness of warps is gaining extensive attentions in image stitching. Recent warps such as SPHP and AANAP, use global similarity warps to mitigate projective distortion (which enlarges regions), however, they necessarily bring in perspective distortion (which generates inconsistencies). In this paper, we propose a novel quasi-homography warp, which effectively balances the perspective distortion against the projective distortion in the non-overlapping region to create a more natural-looking panorama. Our approach formulates the warp as the solution of a bivariate system, where perspecti…
Parallax-Tolerant Image Stitching with Epipolar Displacement Field
School of Automation, Southeast University（东南大学自动化学院）；Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education（教育部复杂工程系统测量与控制重点实验室）
Nanjing University of Science and Technology（南京理工大学）；Czech Technical University in Prague（布拉格捷克理工大学）；SZTAKI（匈牙利科学院计算机科学与控制研究所）
When capturing panoramas, people tend to align their cameras with the vertical axis, i.e., the direction of gravity. Moreover, modern devices, such as smartphones and tablets, are equipped with an IMU (Inertial Measurement Unit) that can measure the gravity vector accurately. Using this prior, the y-axes of the cameras can be aligned or assumed to be already aligned, reducing their relative orientation to 1-DOF (degree of freedom). Exploiting this assumption, we propose new minimal solutions to panoramic image stitching of images taken by cameras with coinciding optical centers, i.e., undergoi…
Parallax-tolerant Image Stitching via Segmentation-guided Multi-homography Warping
College of Information Science and Engineering, Henan University of Technology（河南工业大学信息科学与工程学院）；Hong Kong University of Science and Technology（香港科技大学）；School of Mathematical Sciences, Shenzhen University（深圳大学数学科学学院）
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications（重庆邮电大学人工智能学院）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空与航空工程系）；School of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）
Programmable-Room: Interactive Textured 3D Room Meshes Generation Empowered by Large Language Models
Department of Electronic Engineering, Sogang University（电子工程系，ソガン大学）；Department of Electrical and Electronics Engineering, Pusan National University（电气电子工程系，釜山国立大学）
National Chiao Tung University (NCTU)（国立交通大学）；MediaTek-NCTU Research Center（联发科技-国立交通大学研究中心）
In this paper we propose a new problem scenario in image processing, wide-range image blending, which aims to smoothly merge two different input photos into a panorama by generating novel image content for the intermediate region between them. Although such problem is closely related to the topics of image inpainting, image outpainting, and image blending, none of the approaches from these topics is able to easily address it. We introduce an effective deep-learning model to realize wide-range image blending, where a novel Bidirectional Content Transfer module is proposed to perform the conditi…
