arXiv 2104.085272021-04cs.CV

PARE: Part Attention Regressor for 3D Human Body Estimation

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ETH Zurich（苏黎世联邦理工学院）

Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, Michael J. Black

英文摘要：

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable. To address this, we introduce a soft attention mechanism, called the Part Attention REgressor (PARE), that learns to predict body-part-guided attention masks. We observe that state-of-the-art methods rely on global feature representations, making them sensitive to even small occlusions. In contrast, PARE's part-guided attention mechanism overcomes these issues by exploiting information about the visibility of individual body parts while leveraging information from neighboring body-parts to predict occluded parts. We show qualitatively that PARE learns sensible attention masks, and quantitative evaluation confirms that PARE achieves more accurate and robust reconstruction results than existing approaches on both occlusion-specific and standard benchmarks. The code and data are available for research purposes at {\small \url{https://pare.is.tue.mpg.de/}}

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

STAR: Sparse Trained Articulated Human Body Regressor

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）

The SMPL body model is widely used for the estimation, synthesis, and analysis of 3D human pose and shape. While popular, we show that SMPL has several limitations and introduce STAR, which is quantitatively and qualitatively superior to SMPL. First, SMPL has a huge number of parameters resulting from its use of global blend shapes. These dense pose-corrective offsets relate every vertex on the mesh to all the joints in the kinematic tree, capturing spurious long-range correlations. To address this, we define per-joint pose correctives and learn the subset of mesh vertices that are influenced …

2020-08-19cs.CV

POCO: 带置信度的3D姿态与形状估计

POCO: 3D Pose and Shape Estimation with Confidence

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Inria（法国国家信息与自动化研究所）；École normale supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；University of Amsterdam（阿姆斯特丹大学）

针对现有3D人体姿态与形状回归器无法输出置信度的问题，提出可嵌入任意HPS模型的POCO框架，通过双条件策略输出逐样本不确定性，可用于自举训练和视频姿态补全，同时小幅稳定提升姿态估计精度。

2023-08-24cs.CV

ARTS：基于解耦骨骼表示的半解析回归器用于视频人体网格恢复

ARTS: Semi-Analytical Regressor using Disentangled Skeletal Representations for Human Mesh Recovery from Videos

State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）

ARTS提出利用解耦骨骼表示（关节位置、骨骼长度、人体运动）和半解析回归器（TIK、BSF、MCR）从视频中恢复人体网格，在多个基准上超越现有方法。

2024-10-21cs.CV

3D Human Pose Estimation from a Single Image via Distance Matrix Regression

Institut de Robòtica i Informàtica Industrial (CSIC-UPC)（加泰罗尼亚高级科学研究研究所-加泰罗尼亚理工大学工业机器人技术研究所）

This paper addresses the problem of 3D human pose estimation from a single image. We follow a standard two-step pipeline by first detecting the 2D position of the $N$ body joints, and then using these observations to infer 3D pose. For the first step, we use a recent CNN-based detector. For the second step, most existing approaches perform 2$N$-to-3$N$ regression of the Cartesian joint coordinates. We show that more precise pose estimates can be obtained by representing both the 2D and 3D human poses using $N\times N$ distance matrices, and formulating the problem as a 2D-to-3D distance matrix…

2016-11-28cs.CV

Deep3DPose: Realtime Reconstruction of Arbitrarily Posed Human Bodies from Single RGB Images

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Xmov（魔珐科技）；Zhejiang University（浙江大学）；Texas A&M University（德州农工大学）

We introduce an approach that accurately reconstructs 3D human poses and detailed 3D full-body geometric models from single images in realtime. The key idea of our approach is a novel end-to-end multi-task deep learning framework that uses single images to predict five outputs simultaneously: foreground segmentation mask, 2D joints positions, semantic body partitions, 3D part orientations and uv coordinates (uv map). The multi-task network architecture not only generates more visual cues for reconstruction, but also makes each individual prediction more accurate. The CNN regressor is further c…

2021-06-22cs.CV

DeepPose: Human Pose Estimation via Deep Neural Networks

Google（谷歌）

We propose a method for human pose estimation based on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regressors which results in high precision pose estimates. The approach has the advantage of reasoning about pose in a holistic fashion and has a simple but yet powerful formulation which capitalizes on recent advances in Deep Learning. We present a detailed empirical analysis with state-of-art or better performance on four academic benchmarks of diverse real-world images.

2014-08-20cs.CV

更多相关论文收起更多论文

Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network

City University of Hong Kong（香港城市大学）

We propose an heterogeneous multi-task learning framework for human pose estimation from monocular image with deep convolutional neural network. In particular, we simultaneously learn a pose-joint regressor and a sliding-window body-part detector in a deep network architecture. We show that including the body-part detection task helps to regularize the network, directing it to converge to a good solution. We report competitive and state-of-art results on several data sets. We also empirically show that the learned neurons in the middle layer of our network are tuned to localized body parts.

2014-06-13cs.CVcs.LGcs.NE

监控场景中基于低分辨率点云的人体与头部朝向估计

Body and Head Orientation Estimation from Low-Resolution Point Clouds in Surveillance Settings

UC San Diego（加州大学圣地亚哥分校）

本文提出基于双LiDAR低分辨率点云的监控场景人体与头部朝向估计系统，结合椭圆拟合和神经网络集成，在保护隐私的同时实现可比精度，并首次应用于自闭症行为分析。

2024-09-30cs.CV

Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation

Facebook AI Research（Facebook人工智能研究院）

Differently from 2D image datasets such as COCO, large-scale human datasets with 3D ground-truth annotations are very difficult to obtain in the wild. In this paper, we address this problem by augmenting existing 2D datasets with high-quality 3D pose fits. Remarkably, the resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks such as 3DPW. Additionally, training on our augmented data is straightforward as it does not require to mix multiple and incompatible 2D and 3D datasets or to use complic…

2021-10-22cs.CV

A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image

IHPC, A*STAR（新加坡科技研究局高性能计算研究所）；State University of New York at Buffalo（纽约州立大学布法罗分校）

For 3D hand and body pose estimation task in depth image, a novel anchor-based approach termed Anchor-to-Joint regression network (A2J) with the end-to-end learning ability is proposed. Within A2J, anchor points able to capture global-local spatial context information are densely set on depth image as local regressors for the joints. They contribute to predict the positions of the joints in ensemble way to enhance generalization ability. The proposed 3D articulated pose estimation paradigm is different from the state-of-the-art encoder-decoder based FCN, 3D CNN and point-set based manners. To …

2019-08-27cs.CV

OSSO: Obtaining Skeletal Shape from Outside

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IMATI-CNR（意大利国家研究委员会信息技术与应用数学研究所）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）

We address the problem of inferring the anatomic skeleton of a person, in an arbitrary pose, from the 3D surface of the body; i.e. we predict the inside (bones) from the outside (skin). This has many applications in medicine and biomechanics. Existing state-of-the-art biomechanical skeletons are detailed but do not easily generalize to new subjects. Additionally, computer vision and graphics methods that predict skeletons are typically heuristic, not learned from data, do not leverage the full 3D body surface, and are not validated against ground truth. To our knowledge, our system, called OSS…

2022-04-21cs.CV

IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions

German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Max-Planck Institute for Informatics (MPII)（马克斯·普朗克信息学研究所）；Saarland Informatics Campus（萨尔兰信息学园区）

Can we make virtual characters in a scene interact with their surrounding objects through simple instructions? Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions? Inspired by these questions, we present the first framework to synthesize the full-body motion of virtual human characters performing specified actions with 3D objects placed within their reach. Our system takes textual instructions specifying the objects and the associated intentions of the virtual characters as input and outputs diverse sequences of full-body motions. This contrasts ex…

2023-02-26cs.CVcs.GRcs.LG

深入研究像素对齐特征以实现精确的多视角人体网格恢复

Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery

Tsinghua University（清华大学）

本文提出迭代式 PaFF 框架，通过像素对齐反馈特征与逐顶点融合改进多视角人体网格恢复，并解耦全局位姿，在 Human3.6M 上取得 33.02 MPJPE。

2023-01-15cs.CVeess.IV

ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture

UBC（不列颠哥伦比亚大学）；Microsoft（微软）；EPFL（洛桑联邦理工学院）

The accuracy of monocular 3D human pose estimation depends on the viewpoint from which the image is captured. While freely moving cameras, such as on drones, provide control over this viewpoint, automatically positioning them at the location which will yield the highest accuracy remains an open problem. This is the problem that we address in this paper. Specifically, given a short video sequence, we introduce an algorithm that predicts which viewpoints should be chosen to capture future frames so as to maximize 3D human pose estimation accuracy. The key idea underlying our approach is a method…

2020-06-18cs.CVcs.RO