基于自回归扩散模型的可控运动合成与重建
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
AI总结：
本文提出MoDiff，一种基于自回归扩散模型的可控运动合成与重建方法，集成跨模态Transformer编码器与解码器，并引入基于扩散前向过程的数据dropout，在可控运动合成与高保真重建中优于基线。
摘要：
数据驱动且可控的人体运动合成与预测是活跃的研究领域，在交互媒体和社交机器人中有广泛应用。这些领域在给定过去观测生成多样化运动以及处理不完美姿态方面仍面临挑战。本文提出MoDiff，一种在运动序列上以其他模态的控制上下文为条件的自回归概率扩散模型。该模型集成了跨模态Transformer编码器和基于Transformer的解码器，在捕捉运动与控制模态的时间相关性方面十分有效。我们还引入了一种基于扩散前向过程的新数据dropout方法，以提供更丰富的数据表示和稳健生成。我们展示了MoDiff在可控运动合成方面优于两个基线的性能，并展示了扩散数据dropout在稳健合成及重建接近记录数据的高保真运动方面的优势。
英文摘要：
Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics. Challenges remain in these fields for generating diverse motions given past observations and dealing with imperfect poses. This paper introduces MoDiff, an autoregressive probabilistic diffusion model over motion sequences conditioned on control contexts of other modalities. Our model integrates a cross-modal Transformer encoder and a Transformer-based decoder, which are found effective in capturing temporal correlations in motion and control modalities. We also introduce a new data dropout method based on the diffusion forward process to provide richer data representations and robust generation. We demonstrate the superior performance of MoDiff in controllable motion synthesis for locomotion with respect to two baselines and show the benefits of diffusion data dropout for robust synthesis and reconstruction of high-fidelity motion close to recorded data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
