arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.03335cs.CVeess.IV
收藏

SADIR:用于3D图像重建的形状感知扩散模型

SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction

University of Virginia(弗吉尼亚大学)

Nivetha Jayakumar, Tonmoy Hossain, Miaomiao Zhang

AI总结:

针对现有3D图像重建方法难以保留物体形状结构、易产生伪影的问题,提出基于扩散模型的形状感知网络SADIR,利用形状先验指导重建,在脑和心脏MRI数据集上表现优于基线方法。

摘要:

从有限数量的2D图像进行3D图像重建是计算机视觉与图像分析领域长期存在的挑战。尽管基于深度学习的方法在该领域已取得出色表现,但现有深度网络往往无法有效利用图像中物体的形状结构,导致重建物体的拓扑结构可能无法得到良好保留,进而产生不连续、孔洞或不同部分间连接错位等伪影。\n本文提出一种基于扩散模型的形状感知3D图像重建网络SADIR,以解决上述问题。与以往主要依赖图像强度空间相关性进行3D重建的方法不同,我们的模型利用从训练数据中学到的形状先验指导重建过程。为此,我们构建了一个联合学习网络,可在变形模型下同时学习平均形状,每个重建图像随后被视为该平均形状的一个变形变体。\n我们在脑部和心脏磁共振图像(MRI)上验证了SADIR模型。实验结果表明,我们的方法优于基线方法,重建误差更低,且能更好地保留图像中物体的形状结构。

英文摘要:

3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑