激发想象力：迈向通用目的的“某某放置”
Stimulating Imagination: Towards General-purpose "Something Something Placement"
AI总结：
本文提出SPORT方法，通过大型视觉模型与扩散位姿估计器相结合，仅传递物体类型信息，在无需大规模训练的情况下实现基于“某某”指令的通用物体放置，并可迁移至真实场景。
摘要：
通用目的物体放置是智能通用机器人的一项基础能力：即使在新环境中，也能按照精确的人类指令重新布置物体。本工作致力于利用“某某”指令实现通用目的物体放置。具体来说，我们将整个过程分解为三个部分，包括物体定位、目标想象和机器人控制，并提出了一种名为SPORT的方法。SPORT利用预训练的大型视觉模型对物体进行广泛的语义推理，并学习一个基于扩散的位姿估计器，以确保在三维空间中生成物理真实的结果。这两部分之间仅传递物体类型（可移动物体或参考物体），这带来了两个好处。其一，由于无需针对机器人场景进行特定微调，我们可以充分利用开放集物体识别与定位的强大能力。此外，基于扩散的估计器只需“想象”放置后的物体位姿，而无需其语义信息。因此，训练负担大大减轻，无需大规模训练。目标位姿估计的训练数据在仿真中收集，并使用GPT-4进行标注。实验结果证明了我们方法的有效性。SPORT不仅能对未见过的仿真物体生成有前景的三维目标位姿，还能无缝应用于真实世界环境。
英文摘要：
General-purpose object placement is a fundamental capability of an intelligent generalist robot: being capable of rearranging objects following precise human instructions even in novel environments. This work is dedicated to achieving general-purpose object placement with ``something something'' instructions. Specifically, we break the entire process down into three parts, including object localization, goal imagination and robot control, and propose a method named SPORT. SPORT leverages a pre-trained large vision model for broad semantic reasoning about objects, and learns a diffusion-based pose estimator to ensure physically-realistic results in 3D space. Only object types (movable or reference) are communicated between these two parts, which brings two benefits. One is that we can fully leverage the powerful ability of open-set object recognition and localization since no specific fine-tuning is needed for the robotic scenario. Moreover, the diffusion-based estimator only need to ``imagine" the object poses after the placement, while no necessity for their semantic information. Thus the training burden is greatly reduced and no massive training is required. The training data for the goal pose estimation is collected in simulation and annotated by using GPT-4. Experimental results demonstrate the effectiveness of our approach. SPORT can not only generate promising 3D goal poses for unseen simulated objects, but also be seamlessly applied to real-world settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
