ACDER: Augmented Curiosity-Driven Experience Replay

Curiosity-Driven Experience Prioritization via Density Estimation

In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised learning, but has not yet been thoroughly researched in RL. To address this problem, we propose a novel Curiosity-Driven Prioritization (CDP) framework to encourage the agent to over-sample those trajectories that have rare achieved goal states. The CDP framework mimics the huma…

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the prediction error of a learned dynamics model as an intrinsic reward has been shown to be effective for exploring a number of sparse-reward environments. We present a method that combines hindsight with curiosity-driven exploration and curriculum learning in order to solve the challengin…

想象后见经验回放：面向稀疏奖励任务的好奇心驱动基于模型的学习

Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Tasks

本文提出想象后见经验回放方法，通过想象数据、后见经验回放和好奇心内在奖励解决稀疏奖励多目标任务，在OpenAI Gym Fetch Robotics基准上数据效率平均提升一个数量级。