arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.07268cs.AI
收藏

一种用于辅助决策的联合规划和学习框架

A Joint Planning and Learning Framework for Human-Aided Decision-Making

Auburn University(奥本大学);NVIDIA Corporation(英伟达公司);Maana Inc.(Maana公司)

Daoming Lyu, Fangkai Yang, Bo Liu, Steven Gustafson

摘要:

传统的强化学习(RL)允许智能体仅通过环境奖励来学习策略,具有长而慢的学习曲线,特别是在开始阶段。相反,人类的学习通常要快得多,因为利用了先验知识和一般知识以及多种信息资源。在本文中,我们提出了一个\textbf{P} lanner- \textbf{A} ctor- \textbf{C} ritic架构为hu \textbf{MAN}为中心的规划和学习(\textbf{PACMAN}),其中一个代理使用先验的,高层次的,确定性的符号知识计划目标导向的行动。PACMAN集成了RL的Actor-Critic算法,以微调其对环境奖励和人类反馈的行为。据我们所知,这是第一个统一的框架,其中基于知识的规划,强化学习和人类教学共同促进代理的策略学习。我们的实验表明,PACMAN导致一个显着的跳跃开始在学习的早期阶段,收敛速度快,方差小,是强大的不一致,罕见的,和误导性的反馈。

英文摘要:

Conventional reinforcement learning (RL) allows an agent to learn policies via environmental rewards only, with a long and slow learning curve, especially at the beginning stage. On the contrary, human learning is usually much faster because prior and general knowledge and multiple information resources are utilized. In this paper, we propose a \textbf{P}lanner-\textbf{A}ctor-\textbf{C}ritic architecture for hu\textbf{MAN}-centered planning and learning (\textbf{PACMAN}), where an agent uses prior, high-level, deterministic symbolic knowledge to plan for goal-directed actions. PACMAN integrates Actor-Critic algorithm of RL to fine-tune its behavior towards both environmental rewards and human feedback. To the best our knowledge, This is the first unified framework where knowledge-based planning, RL, and human teaching jointly contribute to the policy learning of an agent. Our experiments demonstrate that PACMAN leads to a significant jump-start at the early stage of learning, converges rapidly and with small variance, and is robust to inconsistent, infrequent, and misleading feedback.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑