神经策略风格迁移
Neural Policy Style Transfer
AI总结：
该研究提出神经策略风格迁移NPST算法，将风格迁移概念拓展至深度强化学习控制策略，通过逆强化学习基于用户演示训练不同Q网络架构的策略，在接球游戏与人形机器人绘画场景中验证了方法的有效性。
摘要：
风格迁移已在美术、自然语言处理和固定轨迹等多个领域被提出。我们将这一概念拓展应用于深度强化学习框架下的控制策略中。每个网络都被训练以最大化期望奖励，该奖励通常编码动作的目标，可被描述为内容。深度神经网络的表达能力使得编码第二项任务成为可能，这一任务可被描述为风格。本文提出神经策略风格迁移（Neural Policy Style Transfer，NPST）算法，用于将一个策略的风格迁移到另一个策略上，同时保留后者的内容。不同策略通过深度Q网络架构定义，这些模型利用逆强化学习方法、基于演示数据进行训练。研究收集了两组不同的用户演示数据，一组用于内容，另一组用于风格，不同风格按照用户演示的定义进行编码。生成的策略是将内容策略和风格策略输入NPST算法后得到的结果。实验在两个场景中开展：一个是受深度强化学习经典Atari游戏启发的接球游戏场景，另一个是基于作者此前研究、采用全尺寸人形机器人的真实绘画场景。本文提出在NPST框架中实现三种不同的Q网络架构（浅层Q网络、深层Q网络和深度循环Q网络）来编码策略，并对比了每种架构在实验中取得的结果。
英文摘要：
Style Transfer has been proposed in a number of fields: fine arts, natural language processing, and fixed trajectories. We scale this concept up to control policies within a Deep Reinforcement Learning infrastructure. Each network is trained to maximize the expected reward, which typically encodes the goal of an action, and can be described as the content. The expressive power of deep neural networks enables encoding a secondary task, which can be described as the style. The Neural Policy Style Transfer (NPST) algorithm is proposed to transfer the style of one policy to another, while maintaining the content of the latter. Different policies are defined via Deep Q-Network architectures. These models are trained using demonstrations through Inverse Reinforcement Learning. Two different sets of user demonstrations are performed, one for content and other for style. Different styles are encoded as defined by user demonstrations. The generated policy is the result of feeding a content policy and a style policy to the NPST algorithm. Experiments are performed in a catch-ball game inspired by the Deep Reinforcement Learning classical Atari games; and a real-world painting scenario with a full-sized humanoid robot, based on previous works of the authors. The implementation of three different Q-Network architectures (Shallow, Deep and Deep Recurrent Q-Network) to encode the policies within the NPST framework is proposed and the results obtained in the experiments with each of these architectures compared.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
