SpawnNet：从预训练网络中学习可泛化的视觉运动技能
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks
AI总结：
针对预训练视觉骨干在策略学习中的泛化瓶颈，提出双流架构 SpawnNet，通过融合多层预训练表征学习鲁棒策略，在模仿学习中显著提升了类别泛化能力。
摘要：
现有的互联网级图像和视频数据集涵盖了广泛的日常物体和任务，为学习在多样化场景中泛化的策略带来了潜力。先前的工作探索了使用不同自监督目标进行视觉预训练。然而，学习到的策略的泛化能力以及相对于微调良好的基线的优势在先前研究中仍不明确。在这项工作中，我们对预训练视觉表征在类别级别的泛化能力进行了针对性研究。我们识别了使用冻结的预训练视觉骨干网络进行策略学习的关键瓶颈，然后提出了 SpawnNet，这是一种新颖的双流架构，学习将预训练的多层表征融合到一个单独的网络中以学习鲁棒的策略。通过大量模拟和真实实验，我们表明在模仿学习设置中，与先前方法相比，具有显著更好的类别泛化能力。开源代码和视频可在我们的网站上找到：https://xingyu-lin.github.io/spawnnet。
英文摘要：
The existing internet-scale image and video datasets cover a wide range of everyday objects and tasks, bringing the potential of learning policies that generalize in diverse scenarios. Prior works have explored visual pre-training with different self-supervised objectives. Still, the generalization capabilities of the learned policies and the advantages over well-tuned baselines remain unclear from prior studies. In this work, we present a focused study of the generalization capabilities of the pre-trained visual representations at the categorical level. We identify the key bottleneck in using a frozen pre-trained visual backbone for policy learning and then propose SpawnNet, a novel two-stream architecture that learns to fuse pre-trained multi-layer representations into a separate network to learn a robust policy. Through extensive simulated and real experiments, we show significantly better categorical generalization compared to prior approaches in imitation learning settings. Open-sourced code and videos can be found on our website: https://xingyu-lin.github.io/spawnnet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
