arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.12189cs.CVcs.LG
收藏

大规模多人三维人体运动预测与场景上下文

Massively Multi-Person 3D Human Motion Forecasting with Scene Context

University of Bonn(波恩大学);University of Goettingen(哥廷根大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所)

Felix B Mueller, Julian Tanke, Juergen Gall

AI总结:

提出场景感知社交Transformer模型(SAST),结合时间卷积与扩散模型,利用场景和周围人信息预测长期多人三维运动,在厨房数据集上优于现有方法。

摘要:

长期三维人体运动预测具有挑战性:人类行为的随机性使得仅从输入序列生成逼真的人体运动变得困难。场景环境信息以及周围人的运动信息可以极大地辅助生成过程。我们提出了一种场景感知的社交Transformer模型(SAST),用于预测长期(10秒)的人体运动。与之前的模型不同,我们的方法能够建模场景中人数和物体数量变化广泛的交互。我们将时间卷积编码器-解码器架构与基于Transformer的瓶颈相结合,从而高效地融合运动和场景信息。我们使用去噪扩散模型对条件运动分布进行建模。我们在Humans in Kitchens数据集上对我们的方法进行了基准测试,该数据集包含同时可见的1到16个人和29到50个物体。我们的模型在真实性和多样性方面,在不同指标和用户研究中均优于其他方法。代码可在https://github.com/felixbmuller/SAST获取。

英文摘要:

Forecasting long-term 3D human motion is challenging: the stochasticity of human behavior makes it hard to generate realistic human motion from the input sequence alone. Information on the scene environment and the motion of nearby people can greatly aid the generation process. We propose a scene-aware social transformer model (SAST) to forecast long-term (10s) human motion motion. Unlike previous models, our approach can model interactions between both widely varying numbers of people and objects in a scene. We combine a temporal convolutional encoder-decoder architecture with a Transformer-based bottleneck that allows us to efficiently combine motion and scene information. We model the conditional motion distribution using denoising diffusion models. We benchmark our approach on the Humans in Kitchens dataset, which contains 1 to 16 persons and 29 to 50 objects that are visible simultaneously. Our model outperforms other approaches in terms of realism and diversity on different metrics and in a user study. Code is available at https://github.com/felixbmuller/SAST.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑