自洽模型和价值观
Self-Consistent Models and Values
摘要：
环境的学习模型为强化学习（RL）代理提供了对环境进行预测的灵活方法。特别是，模型使规划，即使用更多的计算来改善价值函数或政策，而不需要额外的环境交互。在这项工作中，我们研究了一种增强基于模型的强化学习的方法，通过额外鼓励学习模型和价值函数联合\emph{self-consistent}。我们的方法不同于经典的规划方法，如Dyna，它只更新值，以符合模型。我们提出了多个自我一致性更新，评估这些表格和函数近似设置，并发现，适当的选择，自我一致性有助于政策评估和控制。
英文摘要：
Learned models of the environment provide reinforcement learning (RL) agents with flexible ways of making predictions about the environment. In particular, models enable planning, i.e. using more computation to improve value functions or policies, without requiring additional environment interactions. In this work, we investigate a way of augmenting model-based RL, by additionally encouraging a learned model and value function to be jointly \emph{self-consistent}. Our approach differs from classic planning methods such as Dyna, which only update values to be consistent with the model. We propose multiple self-consistency updates, evaluate these in both tabular and function approximation settings, and find that, with appropriate choices, self-consistency helps both policy evaluation and control.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
