哪些物体能帮助我有效行动？关于物理基础的可供性的推理
Which objects help me to act effectively? Reasoning about physically-grounded affordances
AI总结：
本文提出一种基于LLM和VLM对话的开放世界可供性检测方法，通过物理世界扎根考虑机器人具身性和物体属性，实验证明其能有效选择有用物体并提升性能。
摘要：
为了与开放世界进行有效交互，机器人应理解与已知和新颖物体的交互如何帮助它们实现目标。这一理解的关键方面在于检测物体的可供性，可供性代表了通过以各种方式操纵物体所能实现的潜在效果。我们的方法利用大型语言模型（LLMs）和视觉语言模型（VLMs）的对话来实现开放世界的可供性检测。给定预期动作和效果的开放词汇描述，可以找到环境中有用的物体。通过将我们的系统扎根于物理世界，我们考虑了机器人的具身性以及它所遇到的物体的内在属性。在我们的实验中，我们展示了我们的方法能够根据不同的具身性或预期效果产生定制化的输出。该方法能够从一组干扰物中选择一个有用的物体。针对物理属性微调VLM提高了整体性能。这些结果强调了在物理世界中扎根可供性搜索的重要性，即考虑机器人具身性和物体的物理属性。
英文摘要：
For effective interactions with the open world, robots should understand how interactions with known and novel objects help them towards their goal. A key aspect of this understanding lies in detecting an object's affordances, which represent the potential effects that can be achieved by manipulating the object in various ways. Our approach leverages a dialogue of large language models (LLMs) and vision-language models (VLMs) to achieve open-world affordance detection. Given open-vocabulary descriptions of intended actions and effects, the useful objects in the environment are found. By grounding our system in the physical world, we account for the robot's embodiment and the intrinsic properties of the objects it encounters. In our experiments, we have shown that our method produces tailored outputs based on different embodiments or intended effects. The method was able to select a useful object from a set of distractors. Finetuning the VLM for physical properties improved overall performance. These results underline the importance of grounding the affordance search in the physical world, by taking into account robot embodiment and the physical properties of objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
