机器人场景克隆:通过视觉提示编辑推进零样本机器人场景适应在操作中
Robotic Scene Cloning:Advancing Zero-Shot Robotic Scene Adaptation in Manipulation via Visual Prompt Editing
AI总结:
机器人场景克隆通过视觉提示编辑提升零样本机器人场景适应能力,实现跨场景的高效操作性能
摘要:
现代机器人可以在良好训练的环境中执行广泛简单的任务,并适应多样的场景。然而,在现实用户场景中部署预训练的机器人模型仍然具有挑战性,因为它们的零样本能力有限,通常需要大量的现场数据收集。为了解决这个问题,我们提出了机器人场景克隆(RSC),一种新的方法,通过编辑现有的机器人操作轨迹来实现场景特定的适应。RSC通过利用视觉提示机制和精心调制的条件注入模块,实现准确且场景一致的样本生成。不仅转移纹理,而且对视觉提示作出适度的形状适应,RSC在各种物体类型上展示了可靠的任务性能。在各种模拟和现实环境中的实验表明,RSC显著增强了在目标环境中的策略泛化能力。
英文摘要:
Modern robots can perform a wide range of simple tasks and adapt to diverse scenarios in the well-trained environment. However, deploying pre-trained robot models in real-world user scenarios remains challenging due to their limited zero-shot capabilities, often necessitating extensive on-site data collection. To address this issue, we propose Robotic Scene Cloning (RSC), a novel method designed for scene-specific adaptation by editing existing robot operation trajectories. RSC achieves accurate and scene-consistent sample generation by leveraging a visual prompting mechanism and a carefully tuned condition injection module. Not only transferring textures but also performing moderate shape adaptations in response to the visual prompts, RSC demonstrates reliable task performance across a variety of object types. Experiments across various simulated and real-world environments demonstrate that RSC significantly enhances policy generalization in target environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。