arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05818cs.AIcs.DB
收藏

TKG-Thinker: 向通过代理强化学习实现动态推理的时序知识图谱迈进

TKG-Thinker: Towards Dynamic Reasoning over Temporal Knowledge Graphs via Agentic Reinforcement Learning

School of Computer Science, Wuhan University(武汉大学计算机学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)

Zihao Jiang, Miao Peng, Zhenyan Shan, Wenjie Xu, Ben Liu, Gong Chen, Ziqi Gao, Min Peng

AI总结:

TKG-Thinker通过代理强化学习实现动态推理,提升时序知识图谱问答的性能和泛化能力。

摘要:

时序知识图谱问答(TKGQA)旨在通过利用时序知识库来回答时间敏感的问题。尽管大语言模型(LLMs)在TKGQA中展现出显著潜力,但当前的提示策略主要通过两种方式限制了其效果。首先,在复杂的时序约束下,它们容易产生推理幻觉。其次,静态提示限制了模型的自主性和泛化能力,因为它缺乏通过与时序知识图谱(TKGs)环境的动态交互进行优化。为了解决这些限制,我们提出了TKG-Thinker,这是一种具有自主规划和适应性检索能力的新型代理,用于在时序知识图谱上进行推理。具体来说,TKG-Thinker通过双训练策略通过动态多轮交互与TKGs进行深入的时序推理。我们首先应用监督微调(SFT)与思维链数据来培养核心规划能力,然后进入强化学习(RL)阶段,利用多维奖励来在复杂的时序约束下优化推理策略。在三个开源LLM的基准数据集上的实验结果表明,TKG-Thinker实现了最先进的性能,并在复杂的TKGQA设置中表现出强大的泛化能力。

英文摘要:

Temporal knowledge graph question answering (TKGQA) aims to answer time-sensitive questions by leveraging temporal knowledge bases. While Large Language Models (LLMs) demonstrate significant potential in TKGQA, current prompting strategies constrain their efficacy in two primary ways. First, they are prone to reasoning hallucinations under complex temporal constraints. Second, static prompting limits model autonomy and generalization, as it lack optimization through dynamic interaction with temporal knowledge graphs (TKGs) environments. To address these limitations, we propose \textbf{TKG-Thinker}, a novel agent equipped with autonomous planning and adaptive retrieval capabilities for reasoning over TKGs. Specifically, TKG-Thinker performs in-depth temporal reasoning through dynamic multi-turn interactions with TKGs via a dual-training strategy. We first apply Supervised Fine-Tuning (SFT) with chain of thought data to instill core planning capabilities, followed by a Reinforcement Learning (RL) stage that leverages multi-dimensional rewards to refine reasoning policies under intricate temporal constraints. Experimental results on benchmark datasets with three open-source LLMs show that TKG-Thinker achieves state-of-the-art performance and exhibits strong generalization across complex TKGQA settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑