We introduce Spurfies, a novel method for sparse-view surface reconstruction that disentangles appearance and geometry information to utilize local geometry priors trained on synthetic data. Recent research heavily focuses on 3D reconstruction using dense multi-view setups, typically requiring hundreds of images. However, these methods often struggle with few-view scenarios. Existing sparse-view reconstruction techniques often rely on multi-view stereo networks that need to learn joint priors for geometry and appearance from a large amount of data. In contrast, we introduce a neural point representation that disentangles geometry and appearance to train a local geometry prior using a subset of the synthetic ShapeNet dataset only. During inference, we utilize this surface prior as additional constraint for surface and appearance reconstruction from sparse input views via differentiable volume rendering, restricting the space of possible solutions. We validate the effectiveness of our method on the DTU dataset and demonstrate that it outperforms previous state of the art by 35% in surface quality while achieving competitive novel view synthesis quality. Moreover, in contrast to previous works, our method can be applied to larger, unbounded scenes, such as Mip-NeRF 360.
We present an approach for the planar surface reconstruction of a scene from images with limited overlap. This reconstruction task is challenging since it requires jointly reasoning about single image 3D reconstruction, correspondence between images, and the relative camera pose between images. Past work has proposed optimization-based approaches. We introduce a simpler approach, the PlaneFormer, that uses a transformer applied to 3D-aware plane tokens to perform 3D reasoning. Our experiments show that our approach is substantially more effective than prior work, and that several 3D-specific d…
NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images
Graduate School of Information Science and Technology, Osaka University（大阪大学信息科学与技术研究生院）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；National Engineering Research Center of Visual Technology, School of Computer Science, Peking University（北京大学计算机学院视觉技术国家工程研究中心）；Beijing University of Posts and Telecommunications（北京邮电大学）
Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction
HIT (Shenzhen)（哈尔滨工业大学（深圳））；Tsinghua University（清华大学）；NWPU（西北工业大学）；Beijing Institute of Control and Engineering（北京控制与工程研究所）；Technical University of Munich（慕尼黑工业大学）
PhysicsNeRF: Physics-Guided 3D Reconstruction from Sparse Views
Department of Electrical and Computer Engineering, Texas A\&M University, Doha, Qatar（电气与计算机工程系，德克萨斯A&M大学，多哈，卡塔尔）；College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar（科学与工程学院，哈马德·本·卡伊夫大学，多哈，卡塔尔）
PhysicsNeRF提出物理引导的稀疏视图三维重建框架，通过四个约束和紧凑架构，在8视图下达到21.4 dB PSNR，并分析泛化差距，实现物理一致的可泛化表示。
LGDWT-GS: Local and Global Discrete Wavelet-Regularized 3D Gaussian Splatting for Sparse-View Scene Reconstruction
National Institute of Standards and Technology（国家标准与技术研究院）；Department of Physics, Colorado State University（科罗拉多州立大学物理系）；Electrical Engineering Department, University of Colorado（科罗拉多大学电气工程系）
AD-GS: Anatomical Density-guided 3D Gaussian Splatting for Sparse-View Dental CBCT Reconstruction
Beijing University of Posts and Telecommunications（北京邮电大学）；Peking University Third Hospital（北京大学第三医院）；National Institute of Informatics（信息学研究所）；Peking University（北京大学）
In this paper, we proposed a new deep learning based dense monocular SLAM method. Compared to existing methods, the proposed framework constructs a dense 3D model via a sparse to dense mapping using learned surface normals. With single view learned depth estimation as prior for monocular visual odometry, we obtain both accurate positioning and high quality depth reconstruction. The depth and normal are predicted by a single network trained in a tightly coupled manner.Experimental results show that our method significantly improves the performance of visual tracking and depth prediction in comp…
NerT-CA: Efficient Dynamic Reconstruction from Sparse-view X-ray Coronary Angiography
Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, The Netherlands（数学与计算机科学系，埃因霍温理工大学，埃因霍温，荷兰）；Philips Healthcare, Best, The Netherlands（飞利浦医疗，贝斯特，荷兰）；Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands（电子工程系，埃因霍温理工大学，荷兰）
NeRF-CA: Dynamic Reconstruction of X-ray Coronary Angiography with Extremely Sparse-views
Department of Mathematics & Computer Science, Eindhoven University of Technology（数学与计算机科学系，埃因霍温理工大学）；Philips Healthcare（飞利普医疗）；Department of Electrical Engineering, Eindhoven University of Technology（电子工程系，埃因霍温理工大学）；ASML（ASML公司）
3DGR-CAR: Coronary artery reconstruction from ultra-sparse 2D X-ray views with a 3D Gaussians representation
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC)（中国科学技术大学生命科学与医学部生物医学工程学院）；Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advance Research, USTC（中国科学技术大学苏州研究院医学影像、机器人、分析计算与学习中心）
SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers
University of Pittsburgh（匹兹堡大学）；University of Arizona（亚利桑那大学）；Tongji University（同济大学）；University of Central Florida（中佛罗里达大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
The paper studies planar surface reconstruction of indoor scenes from two views with unknown camera poses. While prior approaches have successfully created object-centric reconstructions of many scenes, they fail to exploit other structures, such as planes, which are typically the dominant components of indoor scenes. In this paper, we reconstruct planar surfaces from multiple views, while jointly estimating camera pose. Our experiments demonstrate that our method is able to advance the state of the art of reconstruction from sparse views, on challenging scenes from Matterport3D. Project site:…
The University of Hong Kong（香港大学）；Tencent Games（腾讯游戏）；Texas A&M University（得克萨斯农工大学）
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images. This task becomes more difficult when only sparse images are provided as input, a scenario where existing neural reconstruction approaches usually produce incomplete or distorted results. Moreover, their inability of generalizing to unseen new scenes impedes their application in practice. Contrarily, SparseNeuS can generalize to new scenes and work well with sparse images (as few as 2 or 3). SparseNeuS adopts signed distance function (SDF) as the surface representation,…
With the availability of more powerful computers, iterative reconstruction algorithms are the subject of an ongoing work in the design of more efficient reconstruction algorithms for X-ray computed tomography. In this work, we show how two analytical reconstruction algorithms can be improved by correcting the corresponding reconstructions using a randomized iterative reconstruction algorithm. The combined analytical reconstruction followed by randomized iterative reconstruction can also be viewed as a reconstruction algorithm which, in the experiments we have conducted, uses up to $35\%$ less …
SparseRecon: Neural Implicit Surface Reconstruction from Sparse Views with Feature and Depth Consistencies
School of Software, Tsinghua University（清华大学软件学院）；Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；China Telecom（中国电信）；Kuaishou Technology（快手科技）；Department of Computer Science, Wayne State University（韦恩州立大学计算机科学系）
A framework of demosaicing and superresolution for color filter array (CFA) via residual image reconstruction and sparse representation is presented.Given the intermediate image produced by certain demosaicing and interpolation technique, a residual image between the final reconstruction image and the intermediate image is reconstructed using sparse representation.The final reconstruction image has richer edges and details than that of the intermediate image. Specifically, a generic dictionary is learned from a large set of composite training data composed of intermediate data and residual dat…
It has been recently shown that incorporating priori knowledge significantly improves the performance of basic compressive sensing based approaches. We have managed to successfully exploit this idea for recovering a matrix as a summation of a Low-rank and a Sparse component from compressive measurements. When applied to the problem of construction of 4D Cardiac MR image sequences in real-time from highly under-sampled $k-$space data, our proposed method achieves superior reconstruction quality compared to the other state-of-the-art methods.
This thesis proposes spatio-spectral techniques for hyperspectral image analysis. Adaptive spatio-spectral support and variable exposure hyperspectral imaging is demonstrated to improve spectral reflectance recovery from hyperspectral images. Novel spectral dimensionality reduction techniques have been proposed from the perspective of spectral only and spatio-spectral information preservation. It was found that the joint sparse and joint group sparse hyperspectral image models achieve lower reconstruction error and higher recognition accuracy using only a small subset of bands. Hyperspectral i…
Computer Vision Center（计算机视觉中心）；Universitat de Barcelona（巴塞罗那大学）
In this paper we propose the first deep unsupervised approach in human body reconstruction to estimate body surface from a sparse set of landmarks, so called DeepMurf. We apply a denoising autoencoder to estimate missing landmarks. Then we apply an attention model to estimate body joints from landmarks. Finally, a cascading network is applied to regress parameters of a statistical generative model that reconstructs body. Our set of proposed loss functions allows us to train the network in an unsupervised way. Results on four public datasets show that our approach accurately reconstructs the hu…
Sparis: Neural Implicit Surface Reconstruction of Indoor Scenes from Sparse Views
Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University（北京信息科学与技术国家研究中心（BNRist），清华大学）；School of Software, Tsinghua University（清华大学软件学院）
SERES: Semantic-aware neural reconstruction from sparse views
Department of Automation, Shanghai Jiao Tong University（自动化系，上海交通大学）；Department of Mechanical and Aerospace Engineering, The University of Manchester（机械与航空航天工程系，曼彻斯特大学）；Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong（机械与自动化工程系，香港中文大学）
This paper studies the problem of 3D volumetric reconstruction from two views of a scene with an unknown camera. While seemingly easy for humans, this problem poses many challenges for computers since it requires simultaneously reconstructing objects in the two views while also figuring out their relationship. We propose a new approach that estimates reconstructions, distributions over the camera/object and camera/camera transformations, as well as an inter-view object affinity matrix. This information is then jointly reasoned over to produce the most likely explanation of the scene. We train …
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；Center for Machine Vision and Signal Analysis, University of Oulu（奥卢大学机器视觉与信号分析中心）
Patch-based sparse representation modeling has shown great potential in image compressive sensing (CS) reconstruction. However, this model usually suffers from some limits, such as dictionary learning with great computational complexity, neglecting the relationship among similar patches. In this paper, a group-based sparse representation method with non-convex regularization (GSR-NCR) for image CS reconstruction is proposed. In GSR-NCR, the local sparsity and nonlocal self-similarity of images is simultaneously considered in a unified framework. Different from the previous methods based on spa…
Sparse modeling is one of the efficient techniques for imaging that allows recovering lost information. In this paper, we present a novel iterative phase-retrieval algorithm using a sparse representation of the object amplitude and phase. The algorithm is derived in terms of a constrained maximum likelihood, where the wave field reconstruction is performed using a number of noisy intensity-only observations with a zero-mean additive Gaussian noise. The developed algorithm enables the optimal solution for the object wave field reconstruction. Our goal is an improvement of the reconstruction qua…
Physics-Guided Null-Space Diffusion with Sparse Masking for Corrective Sparse-View CT Reconstruction
School of Information Engineering, Nanchang University, Nanchang, China（信息工程学院，南昌大学，南昌，中国）；School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China（数学与计算机科学学院，南昌大学，南昌，中国）
We present a novel 3D face reconstruction technique that leverages sparse photometric stereo (PS) and latest advances on face registration/modeling from a single image. We observe that 3D morphable faces approach provides a reasonable geometry proxy for light position calibration. Specifically, we develop a robust optimization technique that can calibrate per-pixel lighting direction and illumination at a very high precision without assuming uniform surface albedos. Next, we apply semantic segmentation on input images and the geometry proxy to refine hairy vs. bare skin regions using tailored …
