Dessie：面向从图像估计关节式3D马的形状与姿态的解耦方法

hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognition

In this paper we present our preliminary work on model-based behavioral analysis of horse motion. Our approach is based on the SMAL model, a 3D articulated statistical model of animal shape. We define a novel SMAL model for horses based on a new template, skeleton and shape space learned from $37$ horse toys. We test the accuracy of our hSMAL model in reconstructing a horse from 3D mocap data and images. We apply the hSMAL model to the problem of lameness detection from video, where we fit the model to images to recover 3D pose and train an ST-GCN network on pose data. A comparison with the sa…

MagicPony：在自然环境中学习铰接式3D动物

MagicPony: Learning Articulated 3D Animals in the Wild

提出MagicPony方法，仅从自然环境单视图图像学习预测铰接式动物的3D形状、姿态等，结合隐式-显式表示与知识蒸馏，在重建任务上超越先前工作并展现强泛化能力。

LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery

Creating high-quality articulated 3D models of animals is challenging either via manual creation or using 3D scanning tools. Therefore, techniques to reconstruct articulated 3D objects from 2D images are crucial and highly useful. In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse). Contrary to existing works that rely on pre-defined template shapes, we do not assume any form of 2D or 3D ground-truth annotations, nor do we leverage any multi-view or temporal informa…

利用来自网络图像的关键点伪标签学习关节化形状

Learning Articulated Shape with Keypoint Pseudo-labels from Web Images

提出用50–150张2D关键点标注图像训练类别关键点估计器，并结合网络图像伪标签与数据选择，提升关节化对象单目3D重建性能。

Learning from Synthetic Animals

Despite great success in human parsing, progress for parsing other deformable articulated objects, like animals, is still limited by the lack of labeled data. In this paper, we use synthetic images and ground truth generated from CAD animal models to address this challenge. To bridge the domain gap between real and synthetic images, we propose a novel consistency-constrained semi-supervised learning method (CC-SSL). Our method leverages both spatial and temporal consistencies, to bootstrap weak models trained on synthetic data with unlabeled real images. We demonstrate the effectiveness of our…

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user-clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user-clicked images. Our framework is based on a novel locally-bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly…