RePerformer：从回放至照片级真实重演的沉浸式以人为中心的体积视频
RePerformer: Immersive Human-centric Volumetric Videos from Playback to Photoreal Reperformance
AI总结：
提出 RePerformer 高斯表示，通过运动与外观解耦、Morton 参数化及语义对齐形变迁移，统一实现人体体积视频的高保真回放与新动作照片级重演。
摘要：
以人为中心的体积视频可提供沉浸式自由视点体验，但现有方法要么侧重于回放一般动态场景，要么侧重于驱动人体化身，限制了其重演一般动态场景的能力。本文提出 RePerformer，这是一种新颖的基于 Gaussian 的表示，将高保真以人为中心的体积视频的回放与重演统一起来。具体而言，我们以层级方式将动态场景解耦为在规范空间中关联的 motion Gaussians 与 appearance Gaussians。我们进一步采用基于 Morton 的参数化，将 appearance Gaussians 高效编码为二维位置图和属性图。为增强泛化能力，我们采用 2D CNN 将位置图映射为属性图，这些图可被组装为 appearance Gaussians，以实现动态场景的高保真渲染。对于重演，我们开发了语义感知对齐模块，并在 motion Gaussians 上应用形变迁移，从而支持新动作下的照片级真实渲染。大量实验验证了 RePerformer 的鲁棒性和有效性，并为以人为中心的体积视频中的“先回放后重演”范式建立了新的基准。
英文摘要：
Human-centric volumetric videos offer immersive free-viewpoint experiences, yet existing methods focus either on replaying general dynamic scenes or animating human avatars, limiting their ability to re-perform general dynamic scenes. In this paper, we present RePerformer, a novel Gaussian-based representation that unifies playback and re-performance for high-fidelity human-centric volumetric videos. Specifically, we hierarchically disentangle the dynamic scenes into motion Gaussians and appearance Gaussians which are associated in the canonical space. We further employ a Morton-based parameterization to efficiently encode the appearance Gaussians into 2D position and attribute maps. For enhanced generalization, we adopt 2D CNNs to map position maps to attribute maps, which can be assembled into appearance Gaussians for high-fidelity rendering of the dynamic scenes. For re-performance, we develop a semantic-aware alignment module and apply deformation transfer on motion Gaussians, enabling photo-real rendering under novel motions. Extensive experiments validate the robustness and effectiveness of RePerformer, setting a new benchmark for playback-then-reperformance paradigm in human-centric volumetric videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
