arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.00552cs.ROcs.HC
收藏

基于3D动态场景图的长期人体轨迹预测

Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs

Massachusetts Institute of Technology(麻省理工学院)

Nicolas Gorlo, Lukas Schmid, Luca Carlone

AI总结:

该研究提出基于3D动态场景图和LLM的长期人体轨迹预测方法,通过预测人-环境交互序列引导60秒轨迹预测,新建半合成数据集验证,性能显著优于零样本基线。

摘要:

我们提出了一种适用于室内以人为中心环境的长期人体轨迹预测新方法,该方法对这类环境中的长时程机器人规划至关重要。当前最先进的人体轨迹预测方法受限于其对避障和短期规划的侧重,且无法建模人体与环境的复杂交互。相比之下,我们的方法通过预测人体与环境的交互序列,并利用这些信息引导最长达60秒时程的轨迹预测,克服了上述局限。我们利用大语言模型(Large Language Models, LLMs),通过将场景的丰富上下文信息作为LLM预测的条件,来预测与环境的交互。这些信息以3D Dynamic Scene Graph(3D动态场景图)的形式给出,该图将环境的几何、语义和可通行性编码为分层表示。随后,我们使用基于连续时间马尔可夫链的概率方法,将这些交互序列锚定为人体位置的多模态时空分布。为评估我们的方法,我们引入了一个新的半合成数据集,包含复杂室内环境中的长期人体轨迹,还附带人体-物体交互标注。我们通过全面的实验评估表明,在60秒的时间时程下,与最优的非特权(即以零样本方式在数据集上评估)基线相比,我们的方法平均负对数似然降低了54%,Best-of-20位移误差降低了26.5%。

英文摘要:

We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments. State-of-the-art human trajectory prediction methods are limited by their focus on collision avoidance and short-term planning, and their inability to model complex interactions of humans with the environment. In contrast, our approach overcomes these limitations by predicting sequences of human interactions with the environment and using this information to guide trajectory predictions over a horizon of up to 60s. We leverage Large Language Models (LLMs) to predict interactions with the environment by conditioning the LLM prediction on rich contextual information about the scene. This information is given as a 3D Dynamic Scene Graph that encodes the geometry, semantics, and traversability of the environment into a hierarchical representation. We then ground these interaction sequences into multi-modal spatio-temporal distributions over human positions using a probabilistic approach based on continuous-time Markov Chains. To evaluate our approach, we introduce a new semi-synthetic dataset of long-term human trajectories in complex indoor environments, which also includes annotations of human-object interactions. We show in thorough experimental evaluations that our approach achieves a 54% lower average negative log-likelihood and a 26.5% lower Best-of-20 displacement error compared to the best non-privileged (i.e., evaluated in a zero-shot fashion on the dataset) baselines for a time horizon of 60s.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑