arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.07116cs.AIcs.LGcs.SYeess.SY
收藏

利用人类领域知识对强化学习问题的经验奖励函数建模

Leveraging human Domain Knowledge to model an empirical Reward function for a Reinforcement Learning problem

Persistent Systems Ltd.(持久系统公司)

Dattaraj Rao

摘要:

传统的强化学习(RL)问题依赖于一个详尽的仿真环境,该环境对问题的真实物理进行建模,并通过观察该环境来训练RL代理。在本文中,我们提出了一种新的方法来创建一个环境,通过建模的奖励功能的基础上提取的经验规则,从人类领域的知识系统的研究。利用这个经验奖励函数,我们将建立一个环境和训练代理。我们将首先创建一个环境,模拟通过恒温器设置机舱温度的效果。这通常在RL问题中通过创建具有详细热力学研究的系统的详尽模型来完成。相反,我们提出了一个经验的方法来模拟奖励功能的基础上人类领域的知识。我们将记录一些我们通常在设置恒温器温度时使用的经验法则,并尝试将其建模为我们的奖励函数。将经验人类领域规则建模为RL的奖励函数是本文的独特之处。这是一个连续的行动空间问题,使用深度确定性策略梯度(DDPG)方法,我们将解决最大化的奖励函数。我们将创建一个策略网络,该网络在给定外部温度和湿度的情况下预测最佳温度设定值。

英文摘要:

Traditional Reinforcement Learning (RL) problems depend on an exhaustive simulation environment that models real-world physics of the problem and trains the RL agent by observing this environment. In this paper, we present a novel approach to creating an environment by modeling the reward function based on empirical rules extracted from human domain knowledge of the system under study. Using this empirical rewards function, we will build an environment and train the agent. We will first create an environment that emulates the effect of setting cabin temperature through thermostat. This is typically done in RL problems by creating an exhaustive model of the system with detailed thermodynamic study. Instead, we propose an empirical approach to model the reward function based on human domain knowledge. We will document some rules of thumb that we usually exercise as humans while setting thermostat temperature and try and model these into our reward function. This modeling of empirical human domain rules into a reward function for RL is the unique aspect of this paper. This is a continuous action space problem and using deep deterministic policy gradient (DDPG) method, we will solve for maximizing the reward function. We will create a policy network that predicts optimal temperature setpoint given external temperature and humidity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑