在运动控制中,对于预训练视觉模型,并非所有策略学习方法都同等有效
For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
AI总结:
本研究系统评估14个预训练视觉模型在3类策略学习方法下的表现,发现预训练有效性高度依赖下游算法,并提倡使用VRF和BC等稳健方法,同时发布21任务基准。
摘要:
近年来,利用预训练视觉模型进行运动控制的研究日益受到关注。现有工作主要强调预训练阶段的重要性,但往往忽视了在下游控制特定微调过程中策略学习所扮演的同等重要角色。因此,目前尚不清楚预训练视觉模型在不同控制策略下的有效性是否一致。为弥补这一认知空白,我们对14个预训练视觉模型进行了全面研究,采用了3类不同的策略学习方法,包括强化学习(RL)、通过行为克隆(BC)的模仿学习以及带视觉奖励函数(VRF)的模仿学习。我们的研究得出一系列引人注目的结果,包括发现预训练的有效性高度依赖于下游策略学习算法的选择。我们表明,基于RL方法的传统公认评估具有高度变异性,因此不可靠,并进一步提倡使用更稳健的方法,如VRF和BC。为促进未来对预训练模型及其策略学习方法进行更通用的评估,我们还随工作发布了涵盖3个不同环境中21个任务的基准测试。
英文摘要:
In recent years, increasing attention has been directed to leveraging pre-trained vision models for motor control. While existing works mainly emphasize the importance of this pre-training phase, the arguably equally important role played by downstream policy learning during control-specific fine-tuning is often neglected. It thus remains unclear if pre-trained vision models are consistent in their effectiveness under different control policies. To bridge this gap in understanding, we conduct a comprehensive study on 14 pre-trained vision models using 3 distinct classes of policy learning methods, including reinforcement learning (RL), imitation learning through behavior cloning (BC), and imitation learning with a visual reward function (VRF). Our study yields a series of intriguing results, including the discovery that the effectiveness of pre-training is highly dependent on the choice of the downstream policy learning algorithm. We show that conventionally accepted evaluation based on RL methods is highly variable and therefore unreliable, and further advocate for using more robust methods like VRF and BC. To facilitate more universal evaluations of pre-trained models and their policy learning methods in the future, we also release a benchmark of 21 tasks across 3 different environments alongside our work.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。