将公式编码为深度网络：用于LTL公式的zero-shot执行的强化学习
Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas
摘要：
我们展示了一个强化学习代理，它使用一个组合递归神经网络，作为输入的LTL公式，并确定令人满意的行动。输入LTL公式以前从未见过，但网络执行zero-shot泛化以满足它们。这是RL代理的多任务学习的一种新形式，其中代理从一组不同的任务中学习，并推广到一组新的不同的任务。网络的公式化使这种能力能够推广。我们在两个领域证明了这种能力。在一个符号域中，智能体找到一个被接受的字母序列。在类似Minecraft的环境中，智能体会找到一系列符合公式的动作。虽然以前的工作可以学习可靠地执行一个公式，给出该公式的例子，我们演示了如何可靠地编码所有公式。这可以形成新的多任务代理的基础，这些代理可以发现子任务并在没有任何额外训练的情况下执行它们，以及遵循更复杂的语言命令的代理。这种推广所需的结构是特定于LTL公式的，这就引出了一个有趣的理论问题：神经网络中需要什么结构才能将zero-shot推广到不同的逻辑？
英文摘要：
We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions. The input LTL formulas have never been seen before, yet the network performs zero-shot generalization to satisfy them. This is a novel form of multi-task learning for RL agents where agents learn from one diverse set of tasks and generalize to a new set of diverse tasks. The formulation of the network enables this capacity to generalize. We demonstrate this ability in two domains. In a symbolic domain, the agent finds a sequence of letters that is accepted. In a Minecraft-like environment, the agent finds a sequence of actions that conform to the formula. While prior work could learn to execute one formula reliably given examples of that formula, we demonstrate how to encode all formulas reliably. This could form the basis of new multitask agents that discover sub-tasks and execute them without any additional training, as well as the agents which follow more complex linguistic commands. The structures required for this generalization are specific to LTL formulas, which opens up an interesting theoretical question: what structures are required in neural networks for zero-shot generalization to different logics?
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
