推理时GRU归一化流的随机细化用于实时视频运动转移
Inference-time Stochastic Refinement of GRU-Normalizing Flow for Real-time Video Motion Transfer
AI总结：
本文提出一种在推理时结合随机采样和GRU-NF的改进方法，以提升实时视频运动转移中未来预测的多样性和准确性。
摘要：
实时视频运动转移应用，如沉浸式游戏和基于视觉的异常检测，需要准确且多样的未来预测以支持逼真的合成和在不确定性下的稳健下游决策。为了提高此类序列预测的多样性，我们提出了一种新的推理时细化技术，将门控循环单元-归一化流（GRU-NF）与随机采样方法相结合。虽然GRU-NF可以通过其在时间预测框架内的归一化流整合来捕捉多模分布，但其确定性转换结构可能会限制表达能力。为了解决这个问题，受随机归一化流（SNF）的启发，我们在GRU-NF推理过程中引入马尔可夫链蒙特卡洛（MCMC）步骤，使模型能够探索更丰富的输出空间，并在不重新训练的情况下更好地近似真实数据分布。我们在基于关键点的视频运动转移管道中验证了我们的方法，其中捕捉时间上一致且感知上多样的未来轨迹对于逼真样本和低带宽通信至关重要。实验表明，我们的推理框架，即门控循环单元-随机归一化流（GRU-SNF），在不牺牲准确性的情况下，在更长的预测时间范围内优于GRU-NF，生成多样化的输出。通过在推理过程中注入随机性，我们的方法更有效地捕捉多模行为。这些结果突显了将随机动态与基于流的序列模型结合用于生成时间序列预测的潜力。代码可在：https://github.com/Tasmiah1408028/Inference-Time-Stochastic-Refinement-Of-GRU-NF-For-Real-Time-Video-Motion-Transfer 获取。
英文摘要：
Real-time video motion transfer applications such as immersive gaming and vision-based anomaly detection require accurate yet diverse future predictions to support realistic synthesis and robust downstream decision making under uncertainty. To improve the diversity of such sequential forecasts we propose a novel inference-time refinement technique that combines Gated Recurrent Unit-Normalizing Flows (GRU-NF) with stochastic sampling methods. While GRU-NF can capture multimodal distributions through its integration of normalizing flows within a temporal forecasting framework, its deterministic transformation structure can limit expressivity. To address this, inspired by Stochastic Normalizing Flows (SNF), we introduce Markov Chain Monte Carlo (MCMC) steps during GRU-NF inference, enabling the model to explore a richer output space and better approximate the true data distribution without retraining. We validate our approach in a keypoint-based video motion transfer pipeline, where capturing temporally coherent and perceptually diverse future trajectories is essential for realistic samples and low bandwidth communication. Experiments show that our inference framework, Gated Recurrent Unit- Stochastic Normalizing Flows (GRU-SNF) outperforms GRU-NF in generating diverse outputs without sacrificing accuracy, even under longer prediction horizons. By injecting stochasticity during inference, our approach captures multimodal behavior more effectively. These results highlight the potential of integrating stochastic dynamics with flow-based sequence models for generative time series forecasting. The code is available at: https://github.com/Tasmiah1408028/Inference-Time-Stochastic-Refinement-Of-GRU-NF-For-Real-Time-Video-Motion-Transfer
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
