我们提出了一种适用于室内以人为中心环境的长期人体轨迹预测新方法,该方法对这类环境中的长时程机器人规划至关重要。当前最先进的人体轨迹预测方法受限于其对避障和短期规划的侧重,且无法建模人体与环境的复杂交互。相比之下,我们的方法通过预测人体与环境的交互序列,并利用这些信息引导最长达60秒时程的轨迹预测,克服了上述局限。我们利用大语言模型(Large Language Models, LLMs),通过将场景的丰富上下文信息作为LLM预测的条件,来预测与环境的交互。这些信息以3D Dynamic Scene Graph(3D动态场景图)的形式给出,该图将环境的几何、语义和可通行性编码为分层表示。随后,我们使用基于连续时间马尔可夫链的概率方法,将这些交互序列锚定为人体位置的多模态时空分布。为评估我们的方法,我们引入了一个新的半合成数据集,包含复杂室内环境中的长期人体轨迹,还附带人体-物体交互标注。我们通过全面的实验评估表明,在60秒的时间时程下,与最优的非特权(即以零样本方式在数据集上评估)基线相比,我们的方法平均负对数似然降低了54%,Best-of-20位移误差降低了26.5%。
英文摘要:
We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments. State-of-the-art human trajectory prediction methods are limited by their focus on collision avoidance and short-term planning, and their inability to model complex interactions of humans with the environment. In contrast, our approach overcomes these limitations by predicting sequences of human interactions with the environment and using this information to guide trajectory predictions over a horizon of up to 60s. We leverage Large Language Models (LLMs) to predict interactions with the environment by conditioning the LLM prediction on rich contextual information about the scene. This information is given as a 3D Dynamic Scene Graph that encodes the geometry, semantics, and traversability of the environment into a hierarchical representation. We then ground these interaction sequences into multi-modal spatio-temporal distributions over human positions using a probabilistic approach based on continuous-time Markov Chains. To evaluate our approach, we introduce a new semi-synthetic dataset of long-term human trajectories in complex indoor environments, which also includes annotations of human-object interactions. We show in thorough experimental evaluations that our approach achieves a 54% lower average negative log-likelihood and a 26.5% lower Best-of-20 displacement error compared to the best non-privileged (i.e., evaluated in a zero-shot fashion on the dataset) baselines for a time horizon of 60s.
SIPTraj: Map-Free End-to-End Trajectory Prediction via Physics-Guided Scene Interaction
School of Data Science and Engineering, Xingzhi College, South China Normal University(华南师范大学行知学院数据科学与工程学院);Hong Kong University of Science and Technology(香港科技大学);University of Macau(澳门大学);College of Intelligence Science and Technology, National University of Defense Technology(国防科技大学智能科学学院);School of Electronic Science and Engineering, South China Normal University(华南师范大学电子科学与工程学院)
ExTraCT -- Explainable Trajectory Corrections from language inputs using Textual description of features
Nanyang Technological University (NTU)(南洋理工大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);National Healthcare Group (NHG)(新加坡国立健保集团)
SparScene: Efficient Traffic Scene Representation via Sparse Graph Learning for Large-Scale Trajectory Generation
Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology(决策与控制系统系,电气工程与计算机科学学院,皇家理工学院);iFLYTEK Co., Ltd.(iFLYTEK公司);School of Mechanical and Aerospace Engineering, Nanyang Technological University(机械与航空航天工程学院,南洋理工大学)
nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction
Heinrich Heine University Düsseldorf(杜塞尔多夫海因里希·海涅大学);Bosch Center for Artificial Intelligence(博世人工智能中心);Robert Bosch GmbH(罗伯特·博世有限公司);University of Osnabrück(奥斯纳布吕克大学);Technical University of Munich(慕尼黑工业大学)
Fast and Accurate Multi-Agent Trajectory Prediction For Crowded Unknown Scenes
School of Marine Science and Technology, Northwestern Polytechnical University(西北工业大学海洋科学与技术学院);Tsinghua University(清华大学);University of Victoria(维多利亚大学)
Stabilizing Trajectory Outputs in End-to-End Autonomous Driving via SC-IMM Based Teacher Signals
The CCS Graduate School of Mobility, Korea Advanced Institute of Science and Technology(韩国科学技术院CCS移动出行研究生院);Korea Advanced Institute of Science and Technology(韩国科学技术院)
Stereo-Planner: Learning Stereo-to-Trajectory Planning for Zero-Shot Sim-to-Real Quadrotor Navigation
School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);School of Electrical and Information Engineering, Tianjin University(天津大学电气与信息工程学院)
M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes
School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)(北京通用人工智能研究院通用人工智能国家重点实验室);Center for Vision, Cognition, Learning, and Autonomy (VCLA), Statistics Department, University of California, Los Angeles (UCLA)(加州大学洛杉矶分校视觉、认知、学习与自主中心(VCLA)统计系)
What Did I Learn? Operational Competence Assessment for AI-Based Trajectory Planners
Netherlands Organisation for Applied Scientific Research(荷兰应用科学研究院);Integrated Vehicle Safety Group(集成车辆安全组);Radboud University(拉德堡德大学);Donders Institute for Brain, Cognition and Behaviour(多纳尔斯脑、认知与行为研究所)
Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
University of Macau(澳门大学);University of Macau Advanced Research Institute in Hengqin(澳门大学横琴研究院);The Chinese University of Hong Kong(香港中文大学);The Hong Kong Polytechnic University(香港理工大学);Shanghai Jiao Tong University(上海交通大学);Tongren Hospital(同仁医院);Duke University(杜克大学)
TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Driving
LiAuto Inc(LiAuto公司);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动研究所)