arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.13599cs.AI
收藏

从奖励中学习规则

Learning Rules from Rewards

School of PPLS, University of Edinburgh(爱丁堡大学PPLS学院)

Guillermo Puebla, Leonidas A. A. Doumas

AI总结:

RRTL通过学习关系规则在Atari游戏中实现有效策略,展示强化信号引导结构化表示学习的可行性。

摘要:

人类可以通过利用结构化关系表示在不同领域中灵活地泛化知识。尽管先前研究已展示此类表示如何支持类比推理,但较少了解它们如何被招募以指导适应性行为。我们通过引入关系回归树学习器(RRTL)来填补这一空白,该模型通过在学习过程中选择任务相关的关系来逐步构建策略,以处理结构化关系输入。RRTL基于关系强化学习框架,但与传统方法不同,它专注于地面(即非变体)规则,这些规则指的是特定的对象配置。在三个关系复杂度逐渐增加的Atari游戏中(Breakout、Pong、Demon Attack),模型通过从大量候选关系中识别一小部分相关关系来有效行动。模型的一个比较版本通过使用相对大小值(例如“更多”、“相同”、“更少”)来划分状态空间,比使用逻辑(二元)分割的版本显示出更稳健的学习能力。这些结果提供了一个证明,即强化信号可以指导结构化表示的选择,为理解如何学习和在适应性行为中部署关系知识提供了计算框架。

英文摘要:

Humans can flexibly generalize knowledge across domains by leveraging structured relational representations. While prior research has shown how such representations support analogical reasoning, less is known about how they are recruited to guide adaptive behavior. We address this gap by introducing the Relational Regression Tree Learner (RRTL), a model that incrementally builds policies over structured relational inputs by selecting task-relevant relations during the learning process. RRTL is grounded in the framework of relational reinforcement learning but diverges from traditional approaches by focusing on ground (i.e., non-variabilized) rules that refer to specific object configurations. Across three Atari games of increasing relational complexity (Breakout, Pong, Demon Attack), the model learns to act effectively by identifying a small set of relevant relations from a broad pool of candidate relations. A comparative version of the model, which partitions the state space using relative magnitude values (e.g., "more", "same", "less"), showed more robust learning than a version using logical (binary) splits. These results provide a proof of principle that reinforcement signals can guide the selection of structured representations, offering a computational framework for understanding how relational knowledge is learned and deployed in adaptive behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑