PlayVirtual：用于强化学习的增强循环一致虚拟轨迹
PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning
摘要：
学习良好的特征表示对于深度强化学习（RL）很重要。然而，由于经验有限，强化学习经常会遇到数据效率低下的问题。对于没有经验或经验较少的轨迹（即，状态-动作序列），数据的缺乏限制了它们用于更好的特征学习。在这项工作中，我们提出了一种新的方法，称为PlayVirtual，它增加了周期一致的虚拟轨迹，以提高数据效率的RL功能表示学习。具体而言，PlayVirtual通过动力学模型基于当前状态和动作预测潜在空间中的未来状态，然后通过后向动力学模型预测先前状态，这形成了轨迹循环。在此基础上，我们增加的行动，以产生大量的虚拟状态动作轨迹。在不受groundtruth状态监督的情况下，我们强制执行一个满足循环一致性约束的轨迹，这可以显着提高数据效率。我们在Atari和DeepMind Control Suite基准测试中验证了我们设计的有效性。我们的方法在两个基准上都达到了最先进的性能。
英文摘要：
Learning good feature representations is important for deep reinforcement learning (RL). However, with limited experience, RL often suffers from data inefficiency for training. For un-experienced or less-experienced trajectories (i.e., state-action sequences), the lack of data limits the use of them for better feature learning. In this work, we propose a novel method, dubbed PlayVirtual, which augments cycle-consistent virtual trajectories to enhance the data efficiency for RL feature representation learning. Specifically, PlayVirtual predicts future states in the latent space based on the current state and action by a dynamics model and then predicts the previous states by a backward dynamics model, which forms a trajectory cycle. Based on this, we augment the actions to generate a large amount of virtual state-action trajectories. Being free of groudtruth state supervision, we enforce a trajectory to meet the cycle consistency constraint, which can significantly enhance the data efficiency. We validate the effectiveness of our designs on the Atari and DeepMind Control Suite benchmarks. Our method achieves the state-of-the-art performance on both benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
