arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.02873cs.AIcs.LG
收藏

基于深度神经网络学习转换模型的可扩展规划

Scalable Planning with Deep Neural Network Learned Transition Models

University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(Vector人工智能研究所);Monash University(莫纳什大学)

Ga Wu, Buser Say, Scott Sanner

摘要:

在许多现实世界中的规划问题的因素,混合离散和连续的状态和动作空间,如水库控制,供暖通风,空调,和导航域,这是很难获得一个复杂的非线性动力学模型,管理状态的演变。然而,现代传感器的普遍存在使我们能够从这些复杂系统中的每一个收集大量数据,并建立其状态转换的精确的非线性深度神经网络模型。但控制任务仍然存在一个主要问题-我们如何使用深度网络学习的过渡模型进行规划,而不诉诸蒙特卡罗树搜索和其他黑盒过渡模型技术,这些技术忽略了模型结构,并且不容易扩展到混合离散和连续域?在本文中,我们介绍了两种类型的非线性规划方法,可以利用深度神经网络学习的过渡模型:混合深度MILP规划器(HD-MILP-Plan)和Tensorflow规划器(TF-Plan)。在HD-MILP-Plan中,我们观察到,深度网络的整流线性单元传递函数不仅允许模型学习的更快收敛,而且还允许将深度网络转换模型直接编译为混合线性规划编码。此外,我们还确定了HD-MILP-Plan的深度网络特定优化,这些优化可以提高基本编码的性能,并表明我们可以根据学习到的深度网络进行最佳规划。在TF-Plan中,我们利用自动微分工具和基于GPU的计算的效率,将纯连续规划问题的子类编码为递归神经网络,并通过反向传播直接优化动作。我们比较了这两个计划,并表明TF-计划是能够近似的最佳计划发现HD-MILP-Plan在更少的计算时间…

英文摘要:

In many real-world planning problems with factored, mixed discrete and continuous state and action spaces such as Reservoir Control, Heating Ventilation, and Air Conditioning, and Navigation domains, it is difficult to obtain a model of the complex nonlinear dynamics that govern state evolution. However, the ubiquity of modern sensors allows us to collect large quantities of data from each of these complex systems and build accurate, nonlinear deep neural network models of their state transitions. But there remains one major problem for the task of control -- how can we plan with deep network learned transition models without resorting to Monte Carlo Tree Search and other black-box transition model techniques that ignore model structure and do not easily extend to mixed discrete and continuous domains? In this paper, we introduce two types of nonlinear planning methods that can leverage deep neural network learned transition models: Hybrid Deep MILP Planner (HD-MILP-Plan) and Tensorflow Planner (TF-Plan). In HD-MILP-Plan, we make the critical observation that the Rectified Linear Unit transfer function for deep networks not only allows faster convergence of model learning, but also permits a direct compilation of the deep network transition model to a Mixed-Integer Linear Program encoding. Further, we identify deep network specific optimizations for HD-MILP-Plan that improve performance over a base encoding and show that we can plan optimally with respect to the learned deep networks. In TF-Plan, we take advantage of the efficiency of auto-differentiation tools and GPU-based computation where we encode a subclass of purely continuous planning problems as Recurrent Neural Networks and directly optimize the actions through backpropagation. We compare both planners and show that TF-Plan is able to approximate the optimal plans found by HD-MILP-Plan in less computation time...

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑