SMPLest-X：面向表现力人体姿态与形状估计的极致规模化扩展
SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation
AI总结：
本研究针对表现力人体姿态与形状估计任务，从数据（40个数据集、1000万样本）和模型（ViT-Huge骨干、SMPLest-X等极简架构）两方面规模化扩展，构建通用基础模型，在七个基准上取得SOTA结果，且具备强迁移性，可微调为专用模型。
摘要：
表现力人体姿态与形状估计（EHPS）统一了身体、手部和面部的动作捕捉，具备众多应用场景。尽管已取得令人鼓舞的进展，但当前最先进的方法都聚焦于在受限数据集上训练创新的架构设计。在本研究中，我们探究了将EHPS向通用基础模型家族规模化扩展的影响。1）在数据规模化方面，我们对40个EHPS数据集开展了系统性调研，涵盖了仅在单个数据集上训练的模型无法应对的广泛场景。更重要的是，借助广泛基准测试过程中获得的洞见，我们优化了训练方案，并筛选出能让EHPS能力实现显著跃升的数据集。最终，我们在来自不同数据源的1000万训练样本规模下达到了收益递减点。2）在模型规模化方面，我们利用视觉Transformer（最高采用ViT-Huge作为骨干网络）研究EHPS领域的模型规模缩放定律。为排除算法设计的影响，我们的实验基于两种极简架构：包含手部与面部定位中间步骤的SMPLer-X，以及更为简化的SMPLest-X——后者将网络精简至最核心的结构，凸显了在关节化手部捕捉方面的显著进步。依托大数据与大模型，这些基础模型在各类测试基准上展现出强劲性能，甚至对未见过的环境也具备出色的迁移能力。此外，我们的微调策略可将通用模型转化为专用模型，使其性能得到进一步提升。值得注意的是，我们的基础模型在AGORA、UBody、EgoBody以及我们提出的用于综合手部评估的SynHand数据集等七个基准上，始终取得了最先进的结果。（代码可在https://github.com/wqyin/SMPLest-X获取）。
英文摘要：
Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications. Despite encouraging progress, current state-of-the-art methods focus on training innovative architectural designs on confined datasets. In this work, we investigate the impact of scaling up EHPS towards a family of generalist foundation models. 1) For data scaling, we perform a systematic investigation on 40 EHPS datasets, encompassing a wide range of scenarios that a model trained on any single dataset cannot handle. More importantly, capitalizing on insights obtained from the extensive benchmarking process, we optimize our training scheme and select datasets that lead to a significant leap in EHPS capabilities. Ultimately, we achieve diminishing returns at 10M training instances from diverse data sources. 2) For model scaling, we take advantage of vision transformers (up to ViT-Huge as the backbone) to study the scaling law of model sizes in EHPS. To exclude the influence of algorithmic design, we base our experiments on two minimalist architectures: SMPLer-X, which consists of an intermediate step for hand and face localization, and SMPLest-X, an even simpler version that reduces the network to its bare essentials and highlights significant advances in the capture of articulated hands. With big data and the large model, the foundation models exhibit strong performance across diverse test benchmarks and excellent transferability to even unseen environments. Moreover, our finetuning strategy turns the generalist into specialist models, allowing them to achieve further performance boosts. Notably, our foundation models consistently deliver state-of-the-art results on seven benchmarks such as AGORA, UBody, EgoBody, and our proposed SynHand dataset for comprehensive hand evaluation. (Code is available at: https://github.com/wqyin/SMPLest-X).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
