The field of neural rendering has witnessed significant progress with advancements in generative models and differentiable rendering techniques. Though 2D diffusion has achieved success, a unified 3D diffusion pipeline remains unsettled. This paper introduces a novel framework called LN3Diff++ to address this gap and enable fast, high-quality, and generic conditional 3D generation. Our approach harnesses a 3D-aware architecture and variational autoencoder (VAE) to encode the input image into a structured, compact, and 3D latent space. The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field. Through training a diffusion model on this 3D-aware latent space, our method achieves state-of-the-art performance on ShapeNet for 3D generation and demonstrates superior performance in monocular 3D reconstruction and conditional 3D generation across various datasets. Moreover, it surpasses existing 3D diffusion methods in terms of inference speed, requiring no per-instance optimization. Our proposed LN3Diff presents a significant advancement in 3D generative modeling and holds promise for various applications in 3D vision and graphics tasks.
Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院);Amazon Alexa Science(亚马逊Alexa科学部);Computer Science Department, Boston College(波士顿学院计算机科学系)
DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation
The University of Hong Kong(香港大学);Brown University(布朗大学);Microsoft Research Asia(微软亚洲研究院);Centre for Garment Production Limited, Hong Kong(香港成衣生产中心有限公司);SUSTech(南方科技大学);Texas A&M University(德克萨斯农工大学)
Point Cloud Geometry Scalable Coding Using a Resolution and Quality-conditioned Latents Probability Estimator
University of Padova(帕多瓦大学);Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico(高等技术学院);Universidade de Lisboa(里斯本大学);Politécnico de Leiria(莱里亚理工学院)
CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
Nanjing University(南京大学);Shanghai AI Lab(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);Fudan University(复旦大学);Tongji University(同济大学)
Point Cloud Geometry Scalable Coding with a Quality-Conditioned Latents Probability Estimator
Università di Padova(帕多瓦大学);Instituto Superior Técnico – Universidade de Lisboa(里斯本大学高等技术学院);ESTG – Politécnico de Leiria(莱里亚理工学院技术与管理学院);Instituto de Telecomunicações(电信研究所)
CSMapping: Scalable Crowdsourced Semantic Mapping and Topology Inference for Autonomous Driving
the Department of Electronic and Computer Engineering, the Hong Kong University of Science and Technology, Hong Kong, China(电子与计算机工程系,香港科技大学,香港,中国);the Academy for Engineering and Technology, Fudan University, Shanghai, China(工程与技术学院,复旦大学,上海,中国);Zhuoyu Technology Co., Ltd., Shenzhen, China(珠海优创科技有限公司,深圳,中国)
Progressive Neural Image Compression with Nested Quantization and Latent Ordering
Qualcomm AI Research, Qualcomm Technologies, Inc.(高通技术公司高通人工智能研究院);University of California Irvine(加州大学欧文分校);Qualcomm AI Research, Qualcomm Technologies Netherlands B.V.(高通技术荷兰有限公司高通人工智能研究院)