面向离线多任务强化学习的目标导向技能抽象
Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning
AI总结：
针对离线多任务强化学习中的知识共享难题，提出GO-Skill方法，通过目标导向技能提取与向量量化构建技能库，并利用分层策略学习动态编排技能，在MetaWorld基准上验证了其有效性与通用性。
摘要：
离线多任务强化学习旨在仅利用预先收集的任务混合数据集，无需与环境进行任何在线交互，即可学习一个能够解决多个任务的统一策略。然而，它在跨任务有效共享知识方面面临重大挑战。受人类学习中所观察到的有效知识抽象的启发，我们提出了目标导向技能抽象（GO-Skill），一种旨在提取和利用可复用技能以增强知识迁移和任务性能的新方法。我们的方法通过目标导向的技能提取过程揭示可复用技能，并利用向量量化构建离散技能库。为了缓解广泛适用技能与任务特定技能之间的类别不平衡，我们引入了一个技能增强阶段来精炼提取的技能。此外，我们通过分层策略学习整合这些技能，使得能够构建一个高层策略，动态编排离散技能以完成特定任务。在MetaWorld基准中多样化的机器人操作任务上进行的大量实验证明了GO-Skill的有效性和多功能性。
英文摘要：
Offline multi-task reinforcement learning aims to learn a unified policy capable of solving multiple tasks using only pre-collected task-mixed datasets, without requiring any online interaction with the environment. However, it faces significant challenges in effectively sharing knowledge across tasks. Inspired by the efficient knowledge abstraction observed in human learning, we propose Goal-Oriented Skill Abstraction (GO-Skill), a novel approach designed to extract and utilize reusable skills to enhance knowledge transfer and task performance. Our approach uncovers reusable skills through a goal-oriented skill extraction process and leverages vector quantization to construct a discrete skill library. To mitigate class imbalances between broadly applicable and task-specific skills, we introduce a skill enhancement phase to refine the extracted skills. Furthermore, we integrate these skills using hierarchical policy learning, enabling the construction of a high-level policy that dynamically orchestrates discrete skills to accomplish specific tasks. Extensive experiments on diverse robotic manipulation tasks within the MetaWorld benchmark demonstrate the effectiveness and versatility of GO-Skill.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
