基于智能Agent的自然语言通信调度策略优化
Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents
摘要:
我们调查的任务,学习遵循自然语言的指示,通过联合推理与视觉观察和语言输入。与现有的从演示示教学习(LfD)开始然后使用强化学习(RL)来微调模型参数的方法相比,我们提出了一种新的策略优化算法,该算法动态地调度演示学习和RL。所提出的训练范例提供了比现有方法更有效的探索和更好的泛化。与现有的集成模型相比,基于我们提出的方法的最佳单个模型在块世界环境中大大降低了50%以上的执行错误。为了进一步说明我们的RL算法的探索策略,我们还包括在训练过程中的政策熵的演变系统的研究。
英文摘要:
We investigate the task of learning to follow natural language instructions by jointly reasoning with visual observations and language inputs. In contrast to existing methods which start with learning from demonstrations (LfD) and then use reinforcement learning (RL) to fine-tune the model parameters, we propose a novel policy optimization algorithm which dynamically schedules demonstration learning and RL. The proposed training paradigm provides efficient exploration and better generalization beyond existing methods. Comparing to existing ensemble models, the best single model based on our proposed method tremendously decreases the execution error by over 50% on a block-world environment. To further illustrate the exploration strategy of our RL algorithm, We also include systematic studies on the evolution of policy entropy during training.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。