该做与不该做:通过教学视频学习理想技能
Do's and Don'ts: Learning Desirable Skills with Instruction Videos
AI总结:
针对无监督技能发现学习复杂与安全行为的难题,提出基于教学视频的DoDont算法,通过指令网络区分并加权期望行为,在少量视频下有效习得理想技能并规避不良行为。
摘要:
无监督技能发现是一种旨在无需显式奖励即可获取多样行为的学习范式。然而,它在学习复杂行为方面面临挑战,且常常导致学习到不安全或不良的行为。例如,在各种连续控制任务中,当前的无监督技能发现方法成功学会了站立等基本运动,但在学习行走和跑步等更复杂的动作时却遇到困难。此外,它们可能习得绊倒和翻滚等不安全行为,或导航至坑洞或危险区域等不良位置。为此,我们提出了DoDont(该做与不该做),一种基于指令的技能发现算法,包含两个阶段。首先,在指令学习阶段,DoDont利用无动作的教学视频训练一个指令网络,以区分理想的转换与不理想的转换。然后,在技能学习阶段,指令网络调整技能发现算法的奖励函数,以加权期望的行为。具体来说,我们将指令网络集成到距离最大化的技能发现算法中,其中指令网络充当距离函数。实验表明,在少于8个教学视频的情况下,DoDont在复杂的连续控制任务中有效地学习了期望的行为并避免了不良行为。代码和视频可在https://mynsng.github.io/dodont/获取。
英文摘要:
Unsupervised skill discovery is a learning paradigm that aims to acquire diverse behaviors without explicit rewards. However, it faces challenges in learning complex behaviors and often leads to learning unsafe or undesirable behaviors. For instance, in various continuous control tasks, current unsupervised skill discovery methods succeed in learning basic locomotions like standing but struggle with learning more complex movements such as walking and running. Moreover, they may acquire unsafe behaviors like tripping and rolling or navigate to undesirable locations such as pitfalls or hazardous areas. In response, we present DoDont (Do's and Don'ts), an instruction-based skill discovery algorithm composed of two stages. First, in an instruction learning stage, DoDont leverages action-free instruction videos to train an instruction network to distinguish desirable transitions from undesirable ones. Then, in the skill learning stage, the instruction network adjusts the reward function of the skill discovery algorithm to weight the desired behaviors. Specifically, we integrate the instruction network into a distance-maximizing skill discovery algorithm, where the instruction network serves as the distance function. Empirically, with less than 8 instruction videos, DoDont effectively learns desirable behaviors and avoids undesirable ones across complex continuous control tasks. Code and videos are available at https://mynsng.github.io/dodont/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。