DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis
University of Electronic Science and Technology of China(电子科技大学);Qianyuan Laboratory(钱元实验室);Southwest Jiaotong University(西南交通大学);Independent Researcher(独立研究者)
Xinglong Luo, Ao Luo, Zhengning Wang, Yueqi Yang, Chaoyu Feng, Lei Lei, Bing Zeng, Shuaicheng Liu
Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at https://github.com/boomluo02/DMAligner.
Two-phase Hair Image Synthesis by Self-Enhancing Generative Model
Megvii (Face++) USA(旷视(Face++)美国);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data, CUHK(SZ)(香港中文大学(深圳)深圳大数据研究院)
Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
Beijing Normal University(北京师范大学);University of Chinese Academy of Sciences(中国科学院大学);University of Manchester(曼彻斯特大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Tsinghua University(清华大学)
UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset
State Key Laboratory of Novel Software Technology, Nanjing University, China(南京大学新型软件技术国家重点实验室);vivo Mobile Communication Co., Ltd., China(vivo移动通信有限公司)
GPTDrawer: Enhancing Visual Synthesis through ChatGPT
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);University of California San Diego(加利福尼亚大学圣迭戈分校);Northeastern University(东北大学)
MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes
State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);State Key Laboratory of General Artificial Intelligence, BIGAI(北京智源人工智能研究院通用人工智能国家重点实验室);Tsinghua University(清华大学)
Enhancing Object Coherence in Layout-to-Image Synthesis
School of Computer Science, Fudan University(复旦大学计算机科学学院);College of Computer Science and Technology, Zhejiang University of Technology(浙江工业大学计算机科学与技术学院)
Enhancing Quantitative Image Synthesis through Pretraining and Resolution Scaling for Bone Mineral Density Estimation from a Plain X-ray Image
Nara Institute of Science and Technology(奈良科学技术研究所);Osaka University Graduate School of Medicine(大阪大学医学院);Ehime University Graduate School of Medicine(爱媛大学医学院);CentraleSupélec(中央高等电力学院);Université Paris-Saclay(巴黎-萨克雷大学)
UltraDfeGAN: Detail-Enhancing Generative Adversarial Networks for High-Fidelity Functional Ultrasound Synthesis
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Department of Information and Communication, Institute of Technology of Cambodia(柬埔寨技术学院信息与通信系);Department of Telecommunication and Network Engineering, Institute of Technology of Cambodia(柬埔寨技术学院电信与网络工程系)
Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework
Kennedy Krieger Institute(肯尼迪克里格研究所);Columbia University(哥伦比亚大学);Johns Hopkins University School of Medicine(约翰斯·霍普金斯大学医学院);Johns Hopkins University(约翰斯·霍普金斯大学)
CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Model
Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)(深圳未来网络智能研究院(FNii-深圳));Chinese University of Hong Kong at Shenzhen (CUHKSZ)(香港中文大学(深圳));Sun Yat-sen University(中山大学)
Graph-PiT: Enhancing Structural Coherence in Part-Based Image Synthesis via Graph Priors
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology(广东省超高清沉浸式媒体技术重点实验室);Shenzhen Graduate School, Peking University, China(北京大学深圳研究生院);Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates(穆罕默德·本·扎耶德人工智能大学计算机视觉系)
GeodesicNVS: Probability Density Geodesic Flow Matching for Novel View Synthesis
Huawei Hilbert Research Center (Dresden)(华为希尔伯特研究中心(德累斯顿));Technical University of Munich(慕尼黑工业大学);Karlsruhe University of Applied Sciences(卡尔斯鲁厄应用科学大学)
MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Tencent AI Lab(腾讯人工智能实验室);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong(香港中文大学)
Diffusion Schrödinger Bridge Models for High-Quality MR-to-CT Synthesis for Head and Neck Proton Treatment Planning
Paul Scherrer Institut(保罗·谢勒研究所);ETH Zurich(苏黎世联邦理工学院);University Hospital of Zurich(苏黎世大学医院);Inselspital, Bern University Hospital, University of Bern(伯尔尼大学医院(因塞尔医院),伯尔尼大学)
ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis
Nanyang Technological University(南洋理工大学);University of Oxford(牛津大学);Peking University(北京大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);National University of Singapore(新加坡国立大学)
Enhancing Texture Generation with High-Fidelity Using Advanced Texture Priors
MetaApp AI Research(MetaApp AI研究院);Zhengzhou University(郑州大学);Beijing Institute of Technology(北京理工大学);Beijing University of Technology(北京工业大学);Northeastern University(东北大学)
Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment
Centre for Frontier AI Research (CFAR)(前沿人工智能研究中心);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);Institute of High Performance Computing(高性能计算研究所);XiDian University(西安电子科技大学);National University of Singapore(新加坡国立大学);Hubei University(湖北大学)
Enhancing Novel View Synthesis from extremely sparse views with SfM-free 3D Gaussian Splatting Framework
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(电子与电气工程系,香港理工大学);School of Intelligent Systems Engineering, Sun Yat-sen University(智能系统工程学院,中山大学)