MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Zhejiang University(浙江大学);Key Laboratory of Visual Perception (Zhejiang University), Ministry of Education and Microsoft(视觉感知重点实验室(浙江大学)、教育部与微软);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新区(滨江)区块链与数据安全研究院);University of Washington(华盛顿大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));ZJU-UIUC Institute, Zhejiang University(浙江大学ZJU-UIUC学院)
Zhenyu Zhang, Wenhao Chai, Zhongyu Jiang, Tian Ye, Mingli Song, Jenq-Neng Hwang, Gaoang Wang
Estimating 3D human poses only from a 2D human pose sequence is thoroughly explored in recent years. Yet, prior to this, no such work has attempted to unify 2D and 3D pose representations in the shared feature space. In this paper, we propose \mpm, a unified 2D-3D human pose representation framework via masked pose modeling. We treat 2D and 3D poses as two different modalities like vision and language and build a single-stream transformer-based architecture. We apply two pretext tasks, which are masked 2D pose modeling, and masked 3D pose modeling to pre-train our network and use full-supervision to perform further fine-tuning. A high masking ratio of $71.8~\%$ in total with a spatio-temporal mask sampling strategy leads to better relation modeling both in spatial and temporal domains. \mpm~can handle multiple tasks including 3D human pose estimation, 3D pose estimation from occluded 2D pose, and 3D pose completion in a \textbf{single} framework. We conduct extensive experiments and ablation studies on several widely used human pose datasets and achieve state-of-the-art performance on MPI-INF-3DHP.
Pre-training Point Cloud Compact Model with Partial-aware Reconstruction
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Research Center of Artificial Intelligence, Peng Cheng Laboratory(鹏城实验室人工智能研究中心);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院)
PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modeling
Jiangxi University of Finance and Economics(江西财经大学);Shenzhen University(深圳大学);University of Technology Sydney(悉尼科技大学);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)
LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Institute of Visual Intelligence, Pengcheng Laboratory(鹏城实验室视觉智能研究院);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Bytedance Inc.(字节跳动公司)
Learning semantical dynamics and spatiotemporal collaboration for human pose estimation in video
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
Towards Robust and Realistic Human Pose Estimation via WiFi Signals
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系);Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University(香港理工大学土地测量及地理资讯学系)
RSB-Pose: Robust Short-Baseline Binocular 3D Human Pose Estimation with Occlusion Handling
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Biomedical Engineering, Shanghai Jiao Tong University(上海交通大学生物医学工程学院)
Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoders
National Engineering Research Center for Robot Visual Perception and Control Technology, School of Artificial Intelligence and Robotics, Hunan University(国家机器人视觉感知与控制技术工程研究中心,人工智能与机器人学院,湖南大学);School of Architecture and Art, Central South University(建筑与艺术学院,中南大学);Department of Computer Science and Software Engineering, The University of Western Australia(计算机科学与软件工程系,西澳大学)
HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception
Zhejiang University(浙江大学);Baidu VIS(百度视觉技术部);University of Science and Technology Beijing(北京科技大学);South China University of Technology(华南理工大学);Shanghai Jiao Tong University(上海交通大学);Jilin University(吉林大学)
GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室);X-Era AI Lab(X-Era AI实验室)
MMATR-X: High-Resolution Optical-SAR Joint Pretraining via Conditioned Alignment and Degraded-Target Prediction
College of Electronic Science and Technology, National University of Defense Technology (NUDT)(电子科学与技术学院,国防科技大学);Beijing Institute of Remote Sensing Information(遥感信息研究所)