Fish2Mesh Transformer：从第一人称视角进行三维人体网格恢复
Fish2Mesh Transformer: 3D Human Mesh Recovery from Egocentric Vision
AI总结：
本文提出Fish2Mesh，一种基于鱼眼感知Transformer的第一人称三维人体网格恢复模型，通过第一人称位置嵌入、多任务SMPL回归和弱监督训练，在自遮挡和畸变条件下优于现有方法。
摘要：
第一人称人体估计允许从可穿戴相机的第一人称视角推断用户的身体姿态和体型。尽管已有研究使用姿态估计技术来克服头戴式鱼眼图像引起的自遮挡和图像畸变，但三维人体网格恢复（HMR）技术的类似进展仍然有限。我们提出了Fish2Mesh，这是一个基于鱼眼感知的Transformer模型，专为三维第一人称人体网格恢复而设计。我们提出了一个第一人称位置嵌入模块，为Swin Transformer生成特定于第一人称的位置表，以减少鱼眼图像畸变。我们的模型利用多任务头进行SMPL参数回归和相机平移估计，并估计三维和二维关节作为辅助损失以支持模型训练。为了解决第一人称相机数据稀缺的问题，我们通过使用预训练的4D-Human模型和第三人称相机进行弱监督来创建训练数据集。我们的实验表明，Fish2Mesh优于先前最先进的三维HMR模型。
英文摘要：
Egocentric human body estimation allows for the inference of user body pose and shape from a wearable camera's first-person perspective. Although research has used pose estimation techniques to overcome self-occlusions and image distortions caused by head-mounted fisheye images, similar advances in 3D human mesh recovery (HMR) techniques have been limited. We introduce Fish2Mesh, a fisheye-aware transformer-based model designed for 3D egocentric human mesh recovery. We propose an egocentric position embedding block to generate an ego-specific position table for the Swin Transformer to reduce fisheye image distortion. Our model utilizes multi-task heads for SMPL parametric regression and camera translations, estimating 3D and 2D joints as auxiliary loss to support model training. To address the scarcity of egocentric camera data, we create a training dataset by employing the pre-trained 4D-Human model and third-person cameras for weak supervision. Our experiments demonstrate that Fish2Mesh outperforms previous state-of-the-art 3D HMR models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
