We present a novel method for placing a 3D human animation into a 3D scene while maintaining any human-scene interactions in the animation. We use the notion of computing the most important meshes in the animation for the interaction with the scene, which we call "keyframes." These keyframes allow us to better optimize the placement of the animation into the scene such that interactions in the animations (standing, laying, sitting, etc.) match the affordances of the scene (e.g., standing on the floor or laying in a bed). We compare our method, which we call PAAK, with prior approaches, including POSA, PROX ground truth, and a motion synthesis method, and highlight the benefits of our method with a perceptual study. Human raters preferred our PAAK method over the PROX ground truth data 64.6\% of the time. Additionally, in direct comparisons, the raters preferred PAAK over competing methods including 61.5\% compared to POSA.
Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer
School of Electrical, Electronic and Computer Engineering, University of Western Australia(西澳大学电气、电子与计算机工程学院);University of Melbourne(墨尔本大学);Commonwealth Scientific and Industrial Research Organisation (CSIRO)(联邦科学与工业研究组织);Hunan University(湖南大学);University of Central Florida(中佛罗里达大学)
3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Microsoft Mixed Reality & AI Lab(微软混合现实与人工智能实验室);Technical University of Darmstadt(达姆施塔特工业大学)