As a promising 3D generation technique, multiview diffusion (MVD) has received a lot of attention due to its advantages in terms of generalizability, quality, and efficiency. By finetuning pretrained large image diffusion models with 3D data, the MVD methods first generate multiple views of a 3D object based on an image or text prompt and then reconstruct 3D shapes with multiview 3D reconstruction. However, the sparse views and inconsistent details in the generated images make 3D reconstruction challenging. We present MVD$^2$, an efficient 3D reconstruction method for multiview diffusion (MVD) images. MVD$^2$ aggregates image features into a 3D feature volume by projection and convolution and then decodes volumetric features into a 3D mesh. We train MVD$^2$ with 3D shape collections and MVD images prompted by rendered views of 3D shapes. To address the discrepancy between the generated multiview images and ground-truth views of the 3D shapes, we design a simple-yet-efficient view-dependent training scheme. MVD$^2$ improves the 3D generation quality of MVD and is fast and robust to various MVD methods. After training, it can efficiently decode 3D meshes from multiview images within one second. We train MVD$^2$ with Zero-123++ and ObjectVerse-LVIS 3D dataset and demonstrate its superior performance in generating 3D models from multiview images generated by different MVD methods, using both synthetic and real images as prompts.
EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups
Beijing University of Posts and Telecommunications(北京邮电大学);School of Mechanical Engineering, Shanghai Jiao Tong University(上海交通大学机械工程学院);Xiong’an Aerospace Information Research Institute(雄安航空航天信息研究所);State Key Laboratory of Multimedia Information Processing, School of Computer Science Peking University(北京大学计算机科学学院多媒体信息处理国家重点实验室)
Multiview Compressive Coding for 3D Reconstruction
FAIR, Meta AI(元人工智能公司FAIR实验室)
视觉识别的核心目标是从单个图像中理解对象和场景。由于大规模学习和通用表示,2D识别已经取得了巨大的进展。相比之下,3D提出了新的挑战,源于图像中未描绘的遮挡。先前的工作试图通过从多个视图推断或依赖于稀缺的CAD模型和类别特定的先验来克服这些问题,这些先验阻碍了扩展到新的设置。在这项工作中,我们探索单视图三维重建学习的启发,在自监督学习的进步概括表示。我们介绍了一个简单的框架,该框架在单个对象或整个场景的3D点上操作,再加上来自不同RGB-D视频的类别不可知的大规模训练。我们的模型,多视图压缩编码(MCC),学习压缩输入的外观和几何形状,通过查询3D感知解码器来预测3D结构。MCC的通用性和效率使其能够从大规模和多样化的数据源中学习,并对DALL $\cdot$ E 2想象的新对象或用iPhone在野外捕获的新对象具有很强的泛化能力。
NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images
Graduate School of Information Science and Technology, Osaka University(大阪大学信息科学与技术研究生院);National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);National Engineering Research Center of Visual Technology, School of Computer Science, Peking University(北京大学计算机学院视觉技术国家工程研究中心);Beijing University of Posts and Telecommunications(北京邮电大学)
National ICT Australia (NICTA)(澳大利亚国家信息与通信技术研究院(NICTA));Australian National University(澳大利亚国立大学);Osaka University(大阪大学);Microsoft Research Redmond(微软雷德蒙德研究院);Microsoft, Japan(微软(日本));Microsoft Research Asia(微软亚洲研究院)
MRIo3DS-Net: A Mutually Reinforcing Images to 3D Surface RNN-like framework for model-adaptation indoor 3D reconstruction
Central China Normal University(华中师范大学);Hainan Surveying and Mapping Product Quality Supervision and Inspection Station, MNR(自然资源部海南测绘产品质量监督检验站);Wuhan University(武汉大学)
Large Material Gaussian Model for Relightable 3D Generation
Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);The University of Hong Kong(香港大学);LIGHTSPEED;The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
3DFS: Deformable Dense Depth Fusion and Segmentation for Object Reconstruction from a Handheld Camera
University of Illinois Urbana Champaign(伊利诺伊大学厄巴纳-尚佩恩分校)
我们提出了一种方法,用于从输入视频中对放置在平面上的单个对象进行3D重建和分割。我们的方法是执行密集的深度图估计多个视图使用建议的目标函数,保留细节。然后,使用对估计误差具有鲁棒性的建议的隐式表面函数来融合所得到的深度图,从而产生整个场景的平滑表面重建。最后,使用建议的2D-3D分割,将图像和深度线索与先验和正则化的3D体积和2D分割从剩余的场景中分割出对象。我们在我们的Object-Videos数据集上定性评估3D重建,与融合,多视图立体和分割基线进行比较。我们还使用RGBD场景V2数据集[Henry et al. 2013]定量评估了密集深度估计,并使用Object-Videos数据集的关键帧注释进行了分割。
Multiview Point Cloud Registration Based on Minimum Potential Energy for Free-Form Blade Measurement
College of Electrical and Information Engineering, Hunan University(湖南大学电气与信息工程学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Department of Computer Science and Software Engineering, The University of Western Australia(西澳大学计算机科学与软件工程系)
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
The University of Hong Kong(香港大学);Tencent Games(腾讯游戏);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);University of Pennsylvania(宾夕法尼亚大学);Texas A&M University(得克萨斯农工大学)
Argus: Leveraging Multiview Images for Improved 3-D Scene Understanding With Large Language Models
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Beijing Digital Native Digital City Research Center(北京数字原生数字城市研究院);School of Computing, The University of Georgia(佐治亚大学计算机学院);School of Computer and Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学(深圳)计算机科学与工程系)
GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);University of Science and Technology of China(中国科学技术大学);Cornell University(康奈尔大学)