arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.23022cs.CV
收藏

DMAligner:通过扩散模型基于视角合成增强图像对齐

DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis

University of Electronic Science and Technology of China(电子科技大学);Qianyuan Laboratory(钱元实验室);Southwest Jiaotong University(西南交通大学);Independent Researcher(独立研究者)

Xinglong Luo, Ao Luo, Zhengning Wang, Yueqi Yang, Chaoyu Feng, Lei Lei, Bing Zeng, Shuaicheng Liu

AI总结:

DMAligner通过扩散模型基于视角合成的方法提升图像对齐效果,解决传统光流方法在遮挡和光照变化下的局限性,提出动态感知扩散训练和动态场景图像对齐数据集。

摘要:

图像对齐是计算机视觉中的基础任务,广泛应用于各种领域。现有方法主要采用基于光流的图像变形,但该技术易受遮挡和光照变化等挑战影响,导致对齐质量下降和下游任务准确性降低。本文提出DMAligner,一种基于扩散模型的图像对齐框架,通过视角合成进行对齐。DMAligner从新角度解决图像对齐问题,采用生成方法,具有强大能力并避免流基图像变形的问题。具体而言,我们提出一种动态感知扩散训练方法,用于学习条件图像生成,合成新的视角用于图像对齐。这包括一个动态感知掩码生成(DMP)模块,用于自适应区分动态前景区域和静态背景,使扩散模型更有效地处理经典方法难以解决的挑战。此外,我们开发了动态场景图像对齐(DSIA)数据集,使用Blender生成包含1,033个室内和室外场景的超过30,000对图像,专门用于图像对齐。大量实验结果表明,所提出的方法在DSIA基准测试以及一系列广泛使用的视频数据集上表现优异。我们的代码可在https://github.com/boomluo02/DMAligner上获取。

英文摘要:

Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at https://github.com/boomluo02/DMAligner.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑