从语言生成的演示中学习新技能
Learning Novel Skills from Language-Generated Demonstrations
AI总结：
针对现有机器人学习新技能依赖演示数据或环境交互导致成本高、有安全风险的问题，提出DemoGen框架，通过视觉语言模型和视频扩散模型生成演示，使机器人从自然语言学习新技能，新任务完成率提升至原来三倍。
摘要：
机器人正越来越多地部署在不同领域，以处理需要新技能的任务。然而，当前用于获取新技能的机器人学习算法通常依赖于演示数据集或环境交互，导致劳动力成本高且存在潜在安全风险。为应对这些挑战，本研究提出了DemoGen（一种技能学习框架），使机器人能够从自然语言指令中获取新技能。DemoGen利用视觉语言模型和视频扩散模型生成新技能的演示视频，使机器人能够有效学习新技能。在MetaWorld仿真环境中的实验评估表明，该流程能够生成高保真且可靠的演示。使用生成的演示，各种技能学习算法在新任务上的完成率达到了原来的三倍。这些结果凸显了一种新颖的机器人学习方法，为机器人直观、智能地获取新技能奠定了基础。（项目网站：https://aoqunjin.github.io/LNSLGD/）
英文摘要：
Robots are increasingly deployed across diverse domains to tackle tasks requiring novel skills. However, current robot learning algorithms for acquiring novel skills often rely on demonstration datasets or environment interactions, resulting in high labor costs and potential safety risks. To address these challenges, this study proposes DemoGen, a skill-learning framework that enables robots to acquire novel skills from natural language instructions. DemoGen leverages the vision-language model and the video diffusion model to generate demonstration videos of novel skills, which enabling robots to learn new skills effectively. Experimental evaluations in the MetaWorld simulation environments demonstrate the pipeline's capability to generate high-fidelity and reliable demonstrations. Using the generated demonstrations, various skill learning algorithms achieve an accomplishment rate three times the original on novel tasks. These results highlight a novel approach to robot learning, offering a foundation for the intuitive and intelligent acquisition of novel robotic skills. (Project website: https://aoqunjin.github.io/LNSLGD/)
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
