arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.00845cs.AI
收藏

增强大语言模型在时序点过程中的时间感知能力

Enhancing Temporal Awareness in LLMs for Temporal Point Processes

School of Computer Science and Technology, Tongji University, Shanghai 201804, China(计算机科学与技术学院,同济大学);College of Cyber Security, Jinan University, Guangzhou 510632, China(网络安全学院,暨南大学);Department of Computer Science, University of Illinois Chicago, Chicago, USA(计算机科学系,伊利诺伊大学芝加哥分校)

Lili Chen, Wensheng Gan, Shuang Liang, Philip S. Yu

AI总结:

TPP-TAL通过增强LLMs的时间感知能力,改进了时序点过程中的时间似然估计和事件预测准确性。

摘要:

时序点过程(TPPs)对于分析时间上的事件至关重要,广泛应用于金融、医疗和社会系统等领域。这些过程特别有价值于理解事件随时间展开的方式,考虑其不规则性和依赖性。尽管大规模语言模型(LLMs)在序列建模上取得了成功,但将其应用于时序点过程仍然具有挑战性。关键问题在于当前方法难以有效捕捉时间信息与语义上下文之间的复杂交互,这对准确的事件建模至关重要。在此背景下,我们引入了TPP-TAL(在LLMs中增强时间感知的时序点过程),一种新的即插即用框架,旨在增强LLMs中的时间推理能力。而不是使用传统方法简单地将事件时间和类型嵌入拼接起来,TPP-TAL在将信息输入LLM之前,明确地将时间动态与上下文语义对齐。这种对齐使模型能够更好地感知时间依赖性和事件与其周围上下文之间的长距离交互。通过在几个基准数据集上的全面实验,证明了TPP-TAL在时间似然估计和事件预测准确性方面取得了显著提升,突显了在连续时间事件建模中增强LLMs的时间感知能力的重要性。代码可在https://github.com/chenlilil/TPP-TAL获取。

英文摘要:

Temporal point processes (TPPs) are crucial for analyzing events over time and are widely used in fields such as finance, healthcare, and social systems. These processes are particularly valuable for understanding how events unfold over time, accounting for their irregularity and dependencies. Despite the success of large language models (LLMs) in sequence modeling, applying them to temporal point processes remains challenging. A key issue is that current methods struggle to effectively capture the complex interaction between temporal information and semantic context, which is vital for accurate event modeling. In this context, we introduce TPP-TAL (Temporal Point Processes with Enhanced Temporal Awareness in LLMs), a novel plug-and-play framework designed to enhance temporal reasoning within LLMs. Rather than using the conventional method of simply concatenating event time and type embeddings, TPP-TAL explicitly aligns temporal dynamics with contextual semantics before feeding this information into the LLM. This alignment allows the model to better perceive temporal dependencies and long-range interactions between events and their surrounding contexts. Through comprehensive experiments on several benchmark datasets, it is shown that TPP-TAL delivers substantial improvements in temporal likelihood estimation and event prediction accuracy, highlighting the importance of enhancing temporal awareness in LLMs for continuous-time event modeling. The code is made available at https://github.com/chenlilil/TPP-TAL

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑