通过适配互联网视频知识解决新任务
Solving New Tasks by Adapting Internet Video Knowledge
AI总结：
本文研究将领域内信息与大规模预训练视频模型结合的适配技术，提出逆概率适配策略，以少量示例数据实现机器人新任务的文本条件泛化，并展现对数据质量的鲁棒性。
摘要：
视频生成模型在机器人学中展现出巨大潜力，可作为视觉规划器或策略监督者。当在互联网规模的数据上预训练时，此类视频模型能深入理解与自然语言的对齐，从而通过文本条件促进对新下游行为的泛化。然而，它们可能对智能体所处特定环境的细节不敏感。另一方面，在机器人行为的领域内示例上训练视频模型，自然能编码环境特定的复杂性，但可用演示的规模可能不足以支持通过自然语言规范对未见任务进行泛化。在本工作中，我们研究了将领域内信息与大规模预训练视频模型相结合的不同适配技术，并探讨它们在多大程度上能够实现机器人任务的新文本条件泛化，同时考虑其独立的数据和资源需求。我们成功地在多个机器人环境中证明，使用小规模示例数据适配强大的视频模型可以有效促进对新行为的泛化。特别地，我们提出了一种新颖的适配策略，称为逆概率适配，该策略不仅能在机器人任务和设置中持续实现强泛化性能，而且对适配数据的质量表现出鲁棒性，即使在仅有次优领域内演示的情况下也能成功解决新任务。
英文摘要：
Video generative models demonstrate great promise in robotics by serving as visual planners or as policy supervisors. When pretrained on internet-scale data, such video models intimately understand alignment with natural language, and can thus facilitate generalization to novel downstream behavior through text-conditioning. However, they may not be sensitive to the specificities of the particular environment the agent inhabits. On the other hand, training video models on in-domain examples of robotic behavior naturally encodes environment-specific intricacies, but the scale of available demonstrations may not be sufficient to support generalization to unseen tasks via natural language specification. In this work, we investigate different adaptation techniques that integrate in-domain information with large-scale pretrained video models, and explore the extent to which they enable novel text-conditioned generalization for robotic tasks, while also considering their independent data and resource considerations. We successfully demonstrate across robotic environments that adapting powerful video models with small scales of example data can successfully facilitate generalization to novel behaviors. In particular, we present a novel adaptation strategy, termed Inverse Probabilistic Adaptation, that not only consistently achieves strong generalization performance across robotic tasks and settings, but also exhibits robustness to the quality of adaptation data, successfully solving novel tasks even when only suboptimal in-domain demonstrations are available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
