学习具有数据高效的无监督RL的通用技能策略
Learning Generalizable Skill Policy with Data-Efficient Unsupervised RL
AI总结：
提出GenDa框架，通过技能重标定缓解非平稳性，并引入互补信息瓶颈增强泛化性，实现数据高效且鲁棒的无监督强化学习预训练。
摘要：
无监督强化学习（URL）旨在预训练可扩展的、以技能为条件的策略，无需外部奖励，作为下游控制任务的基础。尽管最近取得了进展，我们认为当前的离策略URL方法受到两个关键且被忽视的瓶颈的限制：（1）非平稳的技能语义和（2）脆弱的泛化性。为了解决这些挑战，我们提出了GenDa（通用数据高效智能体），一个用于鲁棒无监督强化学习的统一框架。首先，我们引入了一种技能重标定机制，以缓解非平稳性并显著提高预训练的数据效率。其次，我们提出了互补信息瓶颈（CIB），鼓励学习到的技能策略关注以自我为中心的特征，并对下游任务的分布偏移具有鲁棒性。通过多种实验，我们证明了GenDa显著增强了URL的可扩展性，具有优越的泛化性和数据效率。我们的代码和视频可在以下网址获取：https://this URL。
英文摘要：
Unsupervised Reinforcement Learning (URL) aims to pre-train scalable, skill-conditioned policies without extrinsic rewards, serving as a foundation for downstream control tasks. Despite recent progress, we argue that current off-policy URL methods are limited by two critical, overlooked bottlenecks: (1) non-stationary skill semantics and (2) brittle generalization. To address these challenges, we propose GenDa (Generalizable Data-efficient Agent), a unified framework for robust unsupervised reinforcement learning. First, we introduce a skill relabeling mechanism to mitigate non-stationarity and significantly improve data efficiency for pre-training. Second, we propose a Complementary Information Bottleneck (CIB), encouraging the learned skill policy to focus on ego-centric features and become robust to distribution shifts for downstream tasks. Through various experiments, we demonstrate that GenDa significantly enhances the scalability of URL with superior generalizability and data efficiency. Our code and videos are available at https://ihatebroccoli.github.io/official-GenDa.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
