基于语义-空间推理的语言条件机器人操作中的物体关系Grounding
Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoning
AI总结：
针对机器人无法处理涉及多物体空间关系复杂指令的问题，提出解耦两阶段方案，通过物体中心语义-空间推理器选择相关物体并融合至可供性学习，有效提升了空间相关指令下的操作性能。
摘要：
物理场景中自然语言的grounded理解能极大有益于遵循人类指令的机器人。在物体操作场景中，现有的端到端模型擅长理解语义概念，但通常无法处理涉及多个物体间空间关系的复杂指令。这需要同时推理物体级别的空间关系并学习精确的像素级操作可供性。我们以解耦的两阶段方案初步应对这一挑战。第一阶段，我们提出以物体为中心的语义-空间推理器，以选择与语言指令任务相关的物体。所选物体的分割结果随后被融合作为可供性学习阶段的额外输入。仅将相关物体的归纳偏置引入视觉-语言可供性学习智能体，就能有效提升其在专为包含空间相关语言指令的物体操作设计的定制实验台中的性能。
英文摘要：
Grounded understanding of natural language in physical scenes can greatly benefit robots that follow human instructions. In object manipulation scenarios, existing end-to-end models are proficient at understanding semantic concepts, but typically cannot handle complex instructions involving spatial relations among multiple objects. which require both reasoning object-level spatial relations and learning precise pixel-level manipulation affordances. We take an initial step to this challenge with a decoupled two-stage solution. In the first stage, we propose an object-centric semantic-spatial reasoner to select which objects are relevant for the language instructed task. The segmentation of selected objects are then fused as additional input to the affordance learning stage. Simply incorporating the inductive bias of relevant objects to a vision-language affordance learning agent can effectively boost its performance in a custom testbed designed for object manipulation with spatial-related language instructions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
