RAM：面向可泛化零样本机器人操作的基于检索的可供性迁移
RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation
AI总结：
本文提出 RAM 框架，通过从多源演示构建可供性记忆并分层检索、将域外2D可供性零样本迁移为域内3D可执行可供性，实现跨物体、环境和具身形态的机器人操作泛化。
摘要：
本工作提出了一种用于零样本机器人操作的检索并迁移框架，名为 RAM，其特点是能够跨多种物体、环境和具身形态实现泛化。与现有方法从昂贵的域内演示中学习操作不同，RAM 利用基于检索的可供性迁移范式，从丰富的域外数据中获得多样化的操作能力。首先，RAM 从多种演示来源（包括机器人数据、人-物交互（HOI）数据和自定义数据）中大规模提取统一的可供性，以构建全面的可供性记忆。随后，在给定语言指令时，RAM 从可供性记忆中分层检索最相似的演示，并以零样本且与具身形态无关的方式，将这种域外二维可供性迁移为域内三维可执行可供性。大量仿真和真实世界评估表明，我们的 RAM 在多种日常任务中始终优于现有工作。此外，RAM 在自动高效数据收集、单样本视觉模仿，以及与 LLM/VLM 集成的长时程操作等下游应用中展现出巨大潜力。更多细节请访问网站 https://yxkryptonite.github.io/RAM/。
英文摘要：
This work proposes a retrieve-and-transfer framework for zero-shot robotic manipulation, dubbed RAM, featuring generalizability across various objects, environments, and embodiments. Unlike existing approaches that learn manipulation from expensive in-domain demonstrations, RAM capitalizes on a retrieval-based affordance transfer paradigm to acquire versatile manipulation capabilities from abundant out-of-domain data. First, RAM extracts unified affordance at scale from diverse sources of demonstrations including robotic data, human-object interaction (HOI) data, and custom data to construct a comprehensive affordance memory. Then given a language instruction, RAM hierarchically retrieves the most similar demonstration from the affordance memory and transfers such out-of-domain 2D affordance to in-domain 3D executable affordance in a zero-shot and embodiment-agnostic manner. Extensive simulation and real-world evaluations demonstrate that our RAM consistently outperforms existing works in diverse daily tasks. Additionally, RAM shows significant potential for downstream applications such as automatic and efficient data collection, one-shot visual imitation, and LLM/VLM-integrated long-horizon manipulation. For more details, please check our website at https://yxkryptonite.github.io/RAM/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
