用于集成2D和3D人体感知的深度多任务架构
Deep Multitask Architecture for Integrated 2D and 3D Human Sensing
摘要：
我们提出了一个深度多任务架构的\emph{fully automatic 2d and 3d human sensing}（DMHS），包括\emph{recognition and reconstruction}，在\emph{monocular images}。该系统计算的图形-背景分割，语义识别人体部位在像素级，并估计的2d和3d姿态的人。该模型通过多任务损失支持所有组件的联合训练，其中早期处理阶段递归地进入高级处理阶段，以实现日益复杂的计算、准确性和鲁棒性。该设计允许我们通过利用多个数据集来绑定一个完整的训练协议，否则这些数据集只会限制性地覆盖一些模型组件：没有身体部位标记的复杂2d图像数据，也没有相关的3d地面实况，或者具有有限2d背景变化的复杂3d数据。在基于几个具有挑战性的2D和3D数据集（LSP，HumanEva，Human3.6M）的详细实验中，我们评估了模型的子结构，各种类型的训练数据在多任务丢失中的效果，并证明了在所有处理级别都可以实现最先进的结果。我们还表明，在野外，我们的单眼RGB架构是感知竞争的最先进的（商业）Kinect系统的基础上RGB-D数据。
英文摘要：
We propose a deep multitask architecture for \emph{fully automatic 2d and 3d human sensing} (DMHS), including \emph{recognition and reconstruction}, in \emph{monocular images}. The system computes the figure-ground segmentation, semantically identifies the human body parts at pixel level, and estimates the 2d and 3d pose of the person. The model supports the joint training of all components by means of multi-task losses where early processing stages recursively feed into advanced ones for increasingly complex calculations, accuracy and robustness. The design allows us to tie a complete training protocol, by taking advantage of multiple datasets that would otherwise restrictively cover only some of the model components: complex 2d image data with no body part labeling and without associated 3d ground truth, or complex 3d data with limited 2d background variability. In detailed experiments based on several challenging 2d and 3d datasets (LSP, HumanEva, Human3.6M), we evaluate the sub-structures of the model, the effect of various types of training data in the multitask loss, and demonstrate that state-of-the-art results can be achieved at all processing levels. We also show that in the wild our monocular RGB architecture is perceptually competitive to a state-of-the art (commercial) Kinect system based on RGB-D data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
