arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.13618cs.CLcs.LG
收藏

基于大语言模型的时间序列事件建模中的时间标记策略

Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models

Capital One;Georgia Institute of Technology(佐治亚理工学院)

Zefang Liu, Nam H. Nguyen, Yinzhu Quan, Shi-Xiong Zhang

AI总结:

本文研究了大语言模型中时间序列事件建模的时间标记策略,通过对比不同编码方法发现,标记器需与数据统计特性匹配才能提升预测性能。

摘要:

在使用大语言模型(LLMs)建模时间事件序列时,表示连续时间是一个关键但研究不足的挑战。已提出多种策略,如字节级表示或日历标记,但最优方法仍不明确,尤其在现实世界事件数据的统计分布差异显著时。本文系统地研究了时间标记方法,比较了不同的编码策略:朴素数字字符串、高精度字节级表示、人类语义日历标记、经典均匀分箱和自适应残差标量量化。我们通过在现实世界数据集上微调LLMs来评估这些策略。分析表明,没有单一策略在所有情况下都优于其他,预测性能严重依赖于标记器与数据统计特性的匹配,突显时间标记作为LLM基于事件建模中的关键但常被忽视的设计维度。

英文摘要:

Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large language models (LLMs). Various strategies like byte-level representations or calendar tokens have been proposed. However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns. This paper presents a systematic empirical study of temporal tokenization for modeling event sequences with LLMs, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar tokens, classic uniform binning, and adaptive residual scalar quantization. We evaluate these strategies by fine-tuning LLMs on real-world datasets that exemplify these diverse distributions. Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, highlighting temporal tokenization as a critical yet often overlooked design dimension in LLM-based event modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑