arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1702.06676cs.LGstat.ML
收藏

摆脱生成模型的反事实控制

Counterfactual Control for Free from Generative Models

Araya(Araya公司);Earth-Life Science Institute(地球生命科学研究所)

Nicholas Guttenberg, Yen Yu, Ryota Kanai

摘要:

我们介绍了一种方法,通过该方法,可以使用生成模型学习动作和未来状态之间的联合分布来自动推断任何期望的奖励函数的控制方案,该奖励函数可以在不重新训练模型的情况下动态改变。在这种方法中,动作选择问题被简化为生成模型的潜在空间上的梯度下降问题,模型本身提供评估结果和找到梯度的方法,就像深度Q网络(DQN)中的奖励网络为动作生成器提供梯度信息一样。与DQN或Actor-Critic不同,它们是特定奖励的条件模型,使用完全联合分布的生成模型允许奖励动态改变。此外,可以检查生成的未来,以了解网络“认为”会发生什么,以及当结果偏离预测时出现了什么问题。

英文摘要:

We introduce a method by which a generative model learning the joint distribution between actions and future states can be used to automatically infer a control scheme for any desired reward function, which may be altered on the fly without retraining the model. In this method, the problem of action selection is reduced to one of gradient descent on the latent space of the generative model, with the model itself providing the means of evaluating outcomes and finding the gradient, much like how the reward network in Deep Q-Networks (DQN) provides gradient information for the action generator. Unlike DQN or Actor-Critic, which are conditional models for a specific reward, using a generative model of the full joint distribution permits the reward to be changed on the fly. In addition, the generated futures can be inspected to gain insight in to what the network 'thinks' will happen, and to what went wrong when the outcomes deviate from prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑