Score distillation sampling (SDS) and its variants have greatly boosted the development of text-to-3D generation, but are vulnerable to geometry collapse and poor textures yet. To solve this issue, we first deeply analyze the SDS and find that its distillation sampling process indeed corresponds to the trajectory sampling of a stochastic differential equation (SDE): SDS samples along an SDE trajectory to yield a less noisy sample which then serves as a guidance to optimize a 3D model. However, the randomness in SDE sampling often leads to a diverse and unpredictable sample which is not always less noisy, and thus is not a consistently correct guidance, explaining the vulnerability of SDS. Since for any SDE, there always exists an ordinary differential equation (ODE) whose trajectory sampling can deterministically and consistently converge to the desired target point as the SDE, we propose a novel and effective "Consistent3D" method that explores the ODE deterministic sampling prior for text-to-3D generation. Specifically, at each training iteration, given a rendered image by a 3D model, we first estimate its desired 3D score function by a pre-trained 2D diffusion model, and build an ODE for trajectory sampling. Next, we design a consistency distillation sampling loss which samples along the ODE trajectory to generate two adjacent samples and uses the less noisy sample to guide another more noisy one for distilling the deterministic prior into the 3D model. Experimental results show the efficacy of our Consistent3D in generating high-fidelity and diverse 3D objects and large-scale scenes, as shown in Fig. 1. The codes are available at https://github.com/sail-sg/Consistent3D.
ReCo-Diff: Residual-Conditioned Deterministic Sampling for Cold Diffusion in Sparse-View CT
National Institute for Mathematical Sciences, Daejeon, 34047, Republic of Korea（韩国全南数学研究所）；Department of Computer Engineering, Korea National University of Transportation, Chungju, 27469, Republic of Korea（韩国交通国立大学计算机工程系）
On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
Institute of Data Science, The University of Hong Kong（数据科学研究所，香港大学）；School of Electronic Information and Communications, Huazhong University of Science and Technology（电子信息与通讯学院，华中科技大学）；University of California, Berkeley（加州大学伯克利分校）；Simons Institute for the Theory of Computing（Simons计算理论研究所）；Institute of Data Science & School of Computing and Data Science, The University of Hong Kong（数据科学研究所及计算与数据科学学院，香港大学）
Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration
Biomedical Engineering Department, College of Future Technology, Peking University（北京大学未来技术学院生物医学工程系）；Institute of Medical Technology, Peking University Health Science Center（北京大学医学部医学技术研究所）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
Simultaneous q-Space Sampling Optimization and Reconstruction for Fast and High-fidelity Diffusion Magnetic Resonance Imaging
University of Chinese Academy of Sciences（中国科学院大学）；State Key Laboratory of Software Development Environment, Beihang University（北京航空航天大学软件开发环境国家重点实验室）；Peng Cheng Laboratory（鹏城实验室）
