在图上明智地行走:基于高效引导-探索的双智能体知识图谱推理
Walk Wisely on Graph: Knowledge Graph Reasoning with Dual Agents via Efficient Guidance-Exploration
AI总结:
针对多跳推理中奖励稀疏和长路径推理困难的问题,提出基于分层强化学习的双智能体模型FULORA,通过高层智能体在简化图上的高效引导与低层智能体在原图上的探索,有效提升了知识图谱推理性能,尤其在长距离推理上表现优异。
摘要:
近年来,多跳推理因其有效性和可解释性而在知识图谱(KG)推理中被广泛研究。然而,以往的多跳推理方法存在两个主要缺陷。首先,由于奖励稀疏,智能体在早期阶段难以学习有效且鲁棒的策略。其次,这些方法在特定数据集(如稀疏知识图谱)上常常失效,此时要求智能体遍历较长的推理路径。为了解决这些问题,我们提出了一种基于分层强化学习(HRL)的双智能体多跳推理模型,命名为FULORA。FULORA通过双智能体之间高效的引导-探索来解决上述推理挑战。高层智能体在简化的知识图谱上行走,为在原始知识图谱上行走的低层智能体提供阶段性提示。在该框架中,低层智能体优化一个平衡两个目标的值函数:(1)最大化回报,以及(2)整合来自高层智能体的高效引导。在三个真实世界知识图谱数据集上进行的实验表明,FULORA优于基于RL的基线方法,尤其是在长距离推理的情况下。
英文摘要:
Recent years, multi-hop reasoning has been widely studied for knowledge graph (KG) reasoning due to its efficacy and interpretability. However, previous multi-hop reasoning approaches are subject to two primary shortcomings. First, agents struggle to learn effective and robust policies at the early phase due to sparse rewards. Second, these approaches often falter on specific datasets like sparse knowledge graphs, where agents are required to traverse lengthy reasoning paths. To address these problems, we propose a multi-hop reasoning model with dual agents based on hierarchical reinforcement learning (HRL), which is named FULORA. FULORA tackles the above reasoning challenges by eFficient GUidance-ExpLORAtion between dual agents. The high-level agent walks on the simplified knowledge graph to provide stage-wise hints for the low-level agent walking on the original knowledge graph. In this framework, the low-level agent optimizes a value function that balances two objectives: (1) maximizing return, and (2) integrating efficient guidance from the high-level agent. Experiments conducted on three real-word knowledge graph datasets demonstrate that FULORA outperforms RL-based baselines, especially in the case of long-distance reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。