FourierHandFlow：使用 Fourier 查询流的神经 4D 手部表示
FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flow
AI总结：
本文提出 FourierHandFlow，用 Fourier 级数表示关节感知查询流并结合 3D 占据场，实现从 RGB 视频学习连续 4D 手部表示，在重建效率与对应相关任务上达到先进水平。
摘要：
近期的 4D 形状表示通过以下方式建模隐式形状的连续时间演化：（1）学习查询流而不利用形状和关节运动先验，或（2）针对每个时间值分别解码形状占据。因此，它们无法有效捕捉关节化形状之间的隐式对应关系，也无法对抖动的时间形变进行正则化。在本文中，我们提出 FourierHandFlow，这是一种面向人手的时空连续表示，将 3D 占据场与表示为 Fourier 级数的关节感知查询流相结合。给定输入 RGB 序列，我们的目标是为每个查询流学习固定数量的 Fourier 系数，以保证平滑且连续的时间形状动态。为有效建模关节化手部的时空形变，我们基于两类 Fourier 查询流构建 4D 表示：（1）姿态流，通过隐式线性混合蒙皮建模受手部关节变化影响的查询动态；（2）形状流，建模逐查询位移流。实验中，我们的方法在基于视频的 4D 重建上取得了最先进结果，同时在计算上比现有 3D/4D 隐式形状表示更高效。我们还利用学到的隐式形状对应关系，展示了运动插值与外推以及纹理迁移结果。据我们所知，FourierHandFlow 是首个从 RGB 视频学习的神经 4D 连续手部表示。代码将公开发布。
英文摘要：
Recent 4D shape representations model continuous temporal evolution of implicit shapes by (1) learning query flows without leveraging shape and articulation priors or (2) decoding shape occupancies separately for each time value. Thus, they do not effectively capture implicit correspondences between articulated shapes or regularize jittery temporal deformations. In this work, we present FourierHandFlow, which is a spatio-temporally continuous representation for human hands that combines a 3D occupancy field with articulation-aware query flows represented as Fourier series. Given an input RGB sequence, we aim to learn a fixed number of Fourier coefficients for each query flow to guarantee smooth and continuous temporal shape dynamics. To effectively model spatio-temporal deformations of articulated hands, we compose our 4D representation based on two types of Fourier query flow: (1) pose flow that models query dynamics influenced by hand articulation changes via implicit linear blend skinning and (2) shape flow that models query-wise displacement flow. In the experiments, our method achieves state-of-the-art results on video-based 4D reconstruction while being computationally more efficient than the existing 3D/4D implicit shape representations. We additionally show our results on motion inter- and extrapolation and texture transfer using the learned correspondences of implicit shapes. To the best of our knowledge, FourierHandFlow is the first neural 4D continuous hand representation learned from RGB videos. The code will be publicly accessible.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
