最近,基于计算机图形学的生成模型的概率反演的视觉问题的多种配方已经被提出。然而,从自然图像的3D感知的应用程序集中在低维的潜在场景,由于建模和推理的挑战。通过逼真的生成模型来解释3D对象形状和2D外观的巨大变化似乎很难,甚至将3D场景与2D图像链接的多对多计算的简单版本进行反演也是如此。本文提出并评估了解决这两个挑战的关键方面的方法。我们表明,它是可能的,以解决具有挑战性的,现实世界的3D视觉问题的近似推理生成模型的图像的基础上渲染的概率CAD(PCAD)程序的输出。我们的PCAD对象几何先验生成对应于合理对象的可变形3D网格,并应用仿射变换将它们放置在场景中。图像似然度基于特征空间中的相似性,该特征空间基于视觉文献中的标准中级图像表示。我们的推理算法集成了单站点和局部阻塞Metropolis-Hastings建议,Hamilton Monte Carlo和歧视性数据驱动的建议,从我们的模型生成的训练数据中学习。我们将这种方法应用于3D人体姿态估计和物体形状重建从单一的图像,实现定量和定性的性能改进超过国家的最先进的基线。
英文摘要:
Recently, multiple formulations of vision problems as probabilistic inversions of generative models based on computer graphics have been proposed. However, applications to 3D perception from natural images have focused on low-dimensional latent scenes, due to challenges in both modeling and inference. Accounting for the enormous variability in 3D object shape and 2D appearance via realistic generative models seems intractable, as does inverting even simple versions of the many-to-many computations that link 3D scenes to 2D images. This paper proposes and evaluates an approach that addresses key aspects of both these challenges. We show that it is possible to solve challenging, real-world 3D vision problems by approximate inference in generative models for images based on rendering the outputs of probabilistic CAD (PCAD) programs. Our PCAD object geometry priors generate deformable 3D meshes corresponding to plausible objects and apply affine transformations to place them in a scene. Image likelihoods are based on similarity in a feature space based on standard mid-level image representations from the vision literature. Our inference algorithm integrates single-site and locally blocked Metropolis-Hastings proposals, Hamiltonian Monte Carlo and discriminative data-driven proposals learned from training data generated from our models. We apply this approach to 3D human pose estimation and object shape reconstruction from single images, achieving quantitative and qualitative performance improvements over state-of-the-art baselines.
Encoding Structural Constraints into Segment Anything Models via Probabilistic Graphical Models
The George Washington University(乔治·华盛顿大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究所,中国科学院);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校)
Probabilistic graphical model based approach for water mapping using GaoFen-2 (GF-2) high resolution imagery and Landsat 8 time series
Tsinghua University(清华大学);Chinese Academy of Sciences(中国科学院);University of California, Berkeley(加州大学伯克利分校);Joint Center for Global Change Studies(全球变化研究联合中心)