arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.04741cs.CVcs.AIcs.GRcs.RO
收藏

基于单幅图像的人体上半身物理似然动画

Physically Plausible Animation of Human Upper Body from a Single Image

National Taiwan University(台湾大学);Stanford University(斯坦福大学)

Ziyuan Huang, Zhengping Zhou, Yung-Yu Chuang, Jiajun Wu, C. Karen Liu

摘要:

我们提出了一种新的方法来生成可控的,动态响应,和真实感的人的动画。给定一个人的图像,我们的系统允许用户使用图像空间中的交互来生成物理上合理的上半身动画(PUBA),例如将他们的手拖到各个位置。我们制定了一个强化学习问题来训练一个动态模型,该模型预测人的下一个2D状态(即,图像上的关键点)以3D动作为条件(即,联合扭矩),以及输出最佳行动以控制人实现预期目标的策略。动态模型利用了3D模拟的表现力和2D视频的视觉真实感。PUBA生成实现任务目标的2D关键点序列,同时对强有力的扰动做出响应。关键点序列然后由姿势到图像生成器转换以产生最终的照片级真实感视频。

英文摘要:

We present a new method for generating controllable, dynamically responsive, and photorealistic human animations. Given an image of a person, our system allows the user to generate Physically plausible Upper Body Animation (PUBA) using interaction in the image space, such as dragging their hand to various locations. We formulate a reinforcement learning problem to train a dynamic model that predicts the person's next 2D state (i.e., keypoints on the image) conditioned on a 3D action (i.e., joint torque), and a policy that outputs optimal actions to control the person to achieve desired goals. The dynamic model leverages the expressiveness of 3D simulation and the visual realism of 2D videos. PUBA generates 2D keypoint sequences that achieve task goals while being responsive to forceful perturbation. The sequences of keypoints are then translated by a pose-to-image generator to produce the final photorealistic video.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑