“摆好它！”：基于组合生成模型的功能性物体布置
"Set It Up!": Functional Object Arrangement with Compositional Generative Models
AI总结：
本文提出SetItUp框架，利用大语言模型和扩散模型组合，从少量示例中学习将欠明确指令转化为功能性物体布置，并在书桌、餐桌和咖啡桌数据集上验证了其优越性。
摘要：
本文研究开发能够理解欠明确指令以创建功能性物体布置的机器人这一挑战，例如“摆一张两人餐桌”；以往的布置方法侧重于远更明确的指令，如“把物体A放到桌子上”。我们引入了一个框架SetItUp，用于学习解释欠明确指令。SetItUp利用少量训练示例和人工编写的程序草图来发现特定场景类型的布置规则。通过利用物体之间抽象空间关系的中间图状表示，SetItUp将布置问题分解为两个子问题：i）从有限数据中学习布置模式，以及ii）将这些抽象关系落实到物体位姿。SetItUp利用大语言模型（LLMs）提出新场景中物体之间的抽象空间关系作为需要满足的约束；然后，它组合一个与这些抽象关系相关联的扩散模型库，以找到满足约束的物体位姿。我们在包含书桌、餐桌和咖啡桌的数据集上验证了我们的框架，结果表明与现有模型相比，在生成物理上合理、功能性强且美观的物体布置方面具有优越性能。
英文摘要：
This paper studies the challenge of developing robots capable of understanding under-specified instructions for creating functional object arrangements, such as "set up a dining table for two"; previous arrangement approaches have focused on much more explicit instructions, such as "put object A on the table." We introduce a framework, SetItUp, for learning to interpret under-specified instructions. SetItUp takes a small number of training examples and a human-crafted program sketch to uncover arrangement rules for specific scene types. By leveraging an intermediate graph-like representation of abstract spatial relationships among objects, SetItUp decomposes the arrangement problem into two subproblems: i) learning the arrangement patterns from limited data and ii) grounding these abstract relationships into object poses. SetItUp leverages large language models (LLMs) to propose the abstract spatial relationships among objects in novel scenes as the constraints to be satisfied; then, it composes a library of diffusion models associated with these abstract relationships to find object poses that satisfy the constraints. We validate our framework on a dataset comprising study desks, dining tables, and coffee tables, with the results showing superior performance in generating physically plausible, functional, and aesthetically pleasing object arrangements compared to existing models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
