MISL 能起飞吗？互信息技能学习的分析与要素
Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning
AI总结：
本文指出 METRA 的优势可在互信息技能学习（MISL）框架内解释，提出新方法 contrastive successor features，以更少组件保留其性能并揭示关键要素。
摘要：
自监督学习有潜力解决当今强化学习的诸多关键挑战，如探索、表征学习和奖励设计。近期工作（METRA）有效论证了放弃互信息转而优化特定 Wasserstein 距离对良好性能至关重要。本文认为，该论文所展现的优势很大程度上可在现有的互信息技能学习（MISL）框架内解释。我们的分析提出了一种新 MISL 方法（contrastive successor features），该方法以更少的活动部件保留了 METRA 的优异性能，并突出了技能学习、对比表征学习与 successor features 间的联系。最后，通过细致的消融研究，我们进一步揭示了我们方法与 METRA 部分关键要素的深层见解。
英文摘要：
Self-supervised learning has the potential of lifting several of the key challenges in reinforcement learning today, such as exploration, representation learning, and reward design. Recent work (METRA) has effectively argued that moving away from mutual information and instead optimizing a certain Wasserstein distance is important for good performance. In this paper, we argue that the benefits seen in that paper can largely be explained within the existing framework of mutual information skill learning (MISL). Our analysis suggests a new MISL method (contrastive successor features) that retains the excellent performance of METRA with fewer moving parts, and highlights connections between skill learning, contrastive representation learning, and successor features. Finally, through careful ablation studies, we provide further insight into some of the key ingredients for both our method and METRA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
