Omnidirectional images (ODIs) are commonly used in real-world visual tasks, and high-resolution ODIs help improve the performance of related visual tasks. Most existing super-resolution methods for ODIs use end-to-end learning strategies, resulting in inferior realness of generated images and a lack of effective out-of-domain generalization capabilities in training methods. Image generation methods represented by diffusion model provide strong priors for visual tasks and have been proven to be effectively applied to image restoration tasks. Leveraging the image priors of the Stable Diffusion (SD) model, we achieve omnidirectional image super-resolution with both fidelity and realness, dubbed as OmniSSR. Firstly, we transform the equirectangular projection (ERP) images into tangent projection (TP) images, whose distribution approximates the planar image domain. Then, we use SD to iteratively sample initial high-resolution results. At each denoising iteration, we further correct and update the initial results using the proposed Octadecaplex Tangent Information Interaction (OTII) and Gradient Decomposition (GD) technique to ensure better consistency. Finally, the TP images are transformed back to obtain the final high-resolution results. Our method is zero-shot, requiring no training or fine-tuning. Experiments of our method on two benchmark datasets demonstrate the effectiveness of our proposed method.
A Comprehensive Comparison of Projections in Omnidirectional Super-Resolution
Beijing Jiaotong University(北京交通大学);Intel Labs China(英特尔中国研究院);Peking University(北京大学);Beijing University of Posts and Telecommunications(北京邮电大学);OPPO Research Institute(OPPO研究院)
OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer
ARC, Tencent PCG(腾讯平台与内容事业群ARC实验室);The University of Tokyo(东京大学);Platform Technologies, Tencent Online Video(腾讯在线视频平台技术部);SIAT, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Shanghai AI Lab(上海人工智能实验室)
D$^{2}$R$^{2}$OSR: Degradation-Disentangled Representation for Real-World Omnidirectional Image Super-Resolution
School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院);ByteDance Inc.(字节跳动公司);School of Computer Science, Peking University(北京大学计算机学院)
Multi-level distortion-aware deformable network for omnidirectional image super-resolution
Department of Electrical and Electronic Engineering(电子与电气工程系);School of Computer Science and Cyber Engineering(计算机科学与网络工程学院);School of Computer Science(计算机科学学院)
Distortion-aware Monocular Depth Estimation for Omnidirectional Images
School of Electronics and Communication Engineering, Sun Yat-sen University(中山大学电子与通信工程学院);University of Western Australia(西澳大利亚大学);Alibaba Group(阿里巴巴集团)
Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting
School of Software, Dalian University of Technology(大连理工大学软件学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems
Hanyang University(汉阳大学)
在本文中,我们提出了一个强大的视觉里程计系统的宽基线相机钻机与宽视场(FOV)鱼眼镜头,它提供了全方位的立体观察环境。为了更鲁棒和准确的自我运动估计,我们在标准VO流水线中添加了三个组件,1)用于改进特征匹配的混合投影模型,2)用于姿态估计的多视图P3P RANSAC算法,以及3)钻机外部参数的在线更新。混合投影模型结合了透视投影和圆柱投影,以最大限度地提高视图之间的重叠,并最大限度地减少图像失真,降低特征匹配性能。多视图P3 P RANSAC算法将传统的P3 P RANSAC扩展到多视图图像,以便在内围值计数中考虑所有视图中的所有特征匹配,以实现鲁棒的姿态估计。最后,在线外部校准无缝集成到后端优化框架中,以便可以自动校正由于冲击或振动引起的相机姿态变化。所提出的系统进行了广泛的评估与地面实况和真实序列的高度动态环境的合成数据集,并证明其优越的性能。