具身空间智能:从隐式场景建模到空间推理
Embodied Spatial Intelligence: from Implicit Scene Modeling to Spatial Reasoning
AI总结:
该研究提出具身空间智能,从场景表征与空间推理两方面弥合大语言模型与物理具身的差距,为能理解自然语言指令的智能机器人奠定基础。
摘要:
本论文提出“具身空间智能”,以应对打造能根据自然语言指令在现实世界中感知并行动的机器人这一挑战。为弥合大语言模型(LLMs)与物理具身之间的差距,我们在场景表征和空间推理两方面提出了研究成果。感知层面,我们利用隐式神经模型构建了鲁棒、可扩展且精准的场景表征,在自监督相机标定、高保真深度场生成和大规模重建方面取得进展。空间推理层面,我们通过提出新型导航基准、将语言锚定到3D的方法以及改进长时程决策的状态反馈机制,增强了LLMs的空间能力。这项工作为机器人能够鲁棒感知周围环境并智能执行复杂语言指令奠定了基础。
英文摘要:
This thesis introduces "Embodied Spatial Intelligence" to address the challenge of creating robots that can perceive and act in the real world based on natural language instructions. To bridge the gap between Large Language Models (LLMs) and physical embodiment, we present contributions on two fronts: scene representation and spatial reasoning. For perception, we develop robust, scalable, and accurate scene representations using implicit neural models, with contributions in self-supervised camera calibration, high-fidelity depth field generation, and large-scale reconstruction. For spatial reasoning, we enhance the spatial capabilities of LLMs by introducing a novel navigation benchmark, a method for grounding language in 3D, and a state-feedback mechanism to improve long-horizon decision-making. This work lays a foundation for robots that can robustly perceive their surroundings and intelligently act upon complex, language-based commands.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。