arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02339cs.CVcs.AIcs.LG
收藏

用于3D人体姿态估计的不确定性感知测试时优化

Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation

State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School(国家一般人工智能实验室,北京大学深圳研究生院);School of Engineering, École Polytechnique Fédérale de Lausanne(瑞士联邦理工学院(洛桑)工程学院);School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne(瑞士联邦理工学院(洛桑)计算机与通信科学学院);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院);Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系);Department of Information, University of Pisa(意大利比萨大学信息系);Department of Information Engineering and Computer Science, University of Trento(意大利特伦托大学信息工程与计算机科学系)

Ti Wang, Mengyuan Liu, Hong Liu, Bin Ren, Yingxuan You, Wenhao Li, Nicu Sebe, Xia Li

AI总结:

针对3D人体姿态估计中的域差距与测试时优化过拟合问题,提出UAO框架,通过关节不确定性约束潜在状态优化,并在多个基准上显著提升性能。

摘要:

尽管数据驱动方法已在3D人体姿态估计中取得成功,但它们常受到域差距影响,并表现出有限的泛化能力。相比之下,基于优化的方法擅长针对具体情况进行微调,但在整体性能上通常不如数据驱动方法。我们观察到,以往基于优化的方法通常依赖投影约束,而该约束只能保证二维空间中的对齐,可能导致过拟合问题。为解决这一问题,我们提出了一个不确定性感知测试时优化(Uncertainty-Aware testing-time Optimization,UAO)框架,该框架保留预训练模型的先验信息,并利用关节的不确定性缓解过拟合问题。具体而言,在训练阶段,我们设计了一个有效的2D到3D网络,用于估计相应的3D姿态,同时量化每个3D关节的不确定性。对于测试期间的优化,所提出的优化框架冻结预训练模型,并且仅优化一个潜在状态。随后采用投影损失,以确保生成的姿态在二维空间中良好对齐,从而实现高质量优化。此外,我们利用每个关节的不确定性来决定每个关节允许进行多大程度的优化。通过在具有挑战性的数据集 Human3.6M、MPI-INF-3DHP 和 3DPW 上进行大量实验,所提出框架的有效性和优越性得到验证。值得注意的是,我们的方法在 Human3.6M 上以5.5%的大幅优势超越了此前最佳结果。代码可在 https://github.com/xiu-cs/UAO-Pose3D 获取。

英文摘要:

Although data-driven methods have achieved success in 3D human pose estimation, they often suffer from domain gaps and exhibit limited generalization. In contrast, optimization-based methods excel in fine-tuning for specific cases but are generally inferior to data-driven methods in overall performance. We observe that previous optimization-based methods commonly rely on a projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem. To address this, we propose an Uncertainty-Aware testing-time Optimization (UAO) framework, which keeps the prior information of the pre-trained model and alleviates the overfitting problem using the uncertainty of joints. Specifically, during the training phase, we design an effective 2D-to-3D network for estimating the corresponding 3D pose while quantifying the uncertainty of each 3D joint. For optimization during testing, the proposed optimization framework freezes the pre-trained model and optimizes only a latent state. Projection loss is then employed to ensure the generated poses are well aligned in 2D space for high-quality optimization. Furthermore, we utilize the uncertainty of each joint to determine how much each joint is allowed for optimization. The effectiveness and superiority of the proposed framework are validated through extensive experiments on challenging datasets: Human3.6M, MPI-INF-3DHP, and 3DPW. Notably, our approach outperforms the previous best result by a large margin of 5.5\% on Human3.6M. Code is available at \href{https://github.com/xiu-cs/UAO-Pose3D}{https://github.com/xiu-cs/UAO-Pose3D}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑