用于基于视频的人体姿态迁移的双向可变形运动调制
Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer
AI总结：
针对视频人体姿态迁移中纹理扭曲、闪烁伪影等问题，提出带双向传播的可变形运动调制方法，提升运动预测与时空一致性，效果优于现有最优方法。
摘要：
基于视频的人体姿态迁移是一种视频到视频的生成任务，其根据一系列目标人体姿态来为一张普通的源人物图像赋予动态效果。考虑到服装上高结构性图案的迁移以及不连续姿态存在的难点，现有方法往往会生成纹理扭曲、闪烁伪影等效果不佳的结果。为解决这些问题，我们提出了一种新颖的可变形运动调制（Deformable Motion Modulation，DMM），该方法利用几何核偏移与自适应权重调制，同时实现特征对齐与风格迁移。与风格迁移中常用的普通风格调制不同，所提出的调制机制通过不规则感受野，根据目标物体形状从风格码中自适应重建出平滑帧。为增强时空一致性，我们利用双向传播，从带噪声姿态生成的扭曲图像序列中提取隐藏运动信息。所提出的特征传播通过前向与后向传播，显著提升了运动预测能力。定量与定性实验结果均表明，该方法在图像保真度与视觉连续性方面优于现有最优方法。源代码已公开于github.com/rocketappslab/bdmm。
英文摘要：
Video-based human pose transfer is a video-to-video generation task that animates a plain source human image based on a series of target human poses. Considering the difficulties in transferring highly structural patterns on the garments and discontinuous poses, existing methods often generate unsatisfactory results such as distorted textures and flickering artifacts. To address these issues, we propose a novel Deformable Motion Modulation (DMM) that utilizes geometric kernel offset with adaptive weight modulation to simultaneously perform feature alignment and style transfer. Different from normal style modulation used in style transfer, the proposed modulation mechanism adaptively reconstructs smoothed frames from style codes according to the object shape through an irregular receptive field of view. To enhance the spatio-temporal consistency, we leverage bidirectional propagation to extract the hidden motion information from a warped image sequence generated by noisy poses. The proposed feature propagation significantly enhances the motion prediction ability by forward and backward propagation. Both quantitative and qualitative experimental results demonstrate superiority over the state-of-the-arts in terms of image fidelity and visual continuity. The source code is publicly available at github.com/rocketappslab/bdmm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
