时间知识图谱(Temporal Knowledge Graph, TKG)推理通常涉及沿时间线补全缺失的事实元素。尽管现有方法能够通过整合时间信息为四元组中的每个事实元素学习良好的嵌入,但它们往往无法推断时间事实的演化。这主要是因为:(1)未能充分探索单个四元组内部的结构和语义关系;(2)未能充分学习不同四元组之间上下文和时间相关性的统一表示。为克服这些局限,我们提出了一种新颖的基于Transformer的TKG推理模型(称为ECEformer),用于学习事件演化链(Evolutionary Chain of Events, ECE)。具体而言,我们按时间顺序展开实体节点的邻域子图,形成事件演化链作为模型输入。随后,我们利用Transformer编码器学习ECE的四元组内部嵌入。接着,我们基于多层感知机(MLP)构建了一个混合上下文推理模块,以学习ECE的四元组间统一表示,同时完成时间知识推理。此外,为增强事件的时效性,我们设计了一个额外的时间预测任务,以在学习到的统一表示中补全有效的时间信息。在六个基准数据集上的大量实验验证了我们方法的最先进性能和有效性。
英文摘要:
Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.
MetaTKG: Learning Evolutionary Meta-Knowledge for Temporal Knowledge Graph Reasoning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs
University of New South Wales(新南威尔士大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Transformer-based Language Models for Reasoning in the Description Logic ALCQ
National and Kapodistrian University of Athens(雅典国立卡波迪斯特里亚大学);Boston University(波士顿大学);Athens Technology Center(雅典技术中心);Archimedes/Athena RC(阿基米德/雅典娜研究中心)
Enhancing Transformers for Generalizable First-Order Logical Entailment
Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China(香港科技大学计算机科学与工程系);Department of Computer Science and Engineering, Beihang University, Beijing, China(北航计算机科学与工程系);Department of Mathematical Sciences, Tsinghua University, Beijing, China(清华大学数学科学系)
Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Zhongguancun Academy(中关村学院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院)
Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs
Australian National University(澳大利亚国立大学);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);Amazon(亚马逊);University of Technology Sydney(悉尼科技大学)
Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformers
Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University(上海交通大学理论科学研究院);School of Mathematical Sciences, Shanghai Jiao Tong University(上海交通大学数学科学学院);School of Mathematics and Statistics, Wuhan University(武汉大学数学学院)
针对 Transformers 多步推理机制不明的问题,提出信息传播规则并通过符号任务进行理论分析,给出 L 层注意力模型单次前向推理步数介于 O(3^{L-1}) 与 O(2^{L-1}) 之间的界限。
Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs
Nanyang Technological University, Singapore(南洋理工大学);The University of Hong Kong, Hong Kong, China(香港大学);The Hong Kong Polytechnic University, Hong Kong, China(香港理工大学);Zhejiang University, Zhejiang, China(浙江大学);Renmin University of China, Beijing, China(中国人民大学);Sun Yat-sen University, Guangdong, China(中山大学);Northeastern University(东北大学);Hebei Key Laboratory of Data Science and Knowledge Management(河北省数据科学与知识管理重点实验室)
CoTEvol: Self-Evolving Chain-of-Thoughts for Data Synthesis in Mathematical Reasoning
Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院);The Chinese University of Hong Kong(香港中文大学);Monash University(墨尔本大学);Independent Researcher(独立研究者);Shanghai Academy of AI for Science(上海人工智能科学研究院)
Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing
Anhui University(安徽大学);Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education(教育部智能计算与信号处理重点实验室);Guangdong University of Foreign Studies(广东外语外贸大学)
Case-Based or Rule-Based: How Do Transformers Do the Math?
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);National Key Laboratory of General Artificial Intelligence, BIGAI(北京智源人工智能研究院通用人工智能国家重点实验室)