arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.16563cs.CL
收藏

评估大语言模型的时间序列特征理解能力:综合分类体系与基准

Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark

JP. Morgan AI Research(摩根大通人工智能研究院)

Elizabeth Fons, Rachneet Kaur, Soham Palande, Zhen Zeng, Tucker Balch, Manuela Veloso, Svitlana Vyetrenko

AI总结:

该研究提出评估大语言模型时间序列理解能力的框架与特征分类体系,合成配套数据集开展实验,揭示了现有模型的优劣势及对数据格式等因素的敏感性。

摘要:

大语言模型(LLMs)为自动时间序列分析与报告提供了可能,这是医疗、金融、气候、能源等诸多领域的关键任务。本文提出一个用于严格评估LLMs时间序列理解能力的框架,覆盖单变量与多变量两种形式。我们引入了一套全面的时间序列特征分类体系,这一关键框架明确界定了时间序列数据固有的各类特征。依托该分类体系,我们系统设计并合成了一个多样化的时间序列数据集,涵盖上述所有不同特征,每个样本都配有文本描述。该数据集为评估LLMs的时间序列理解水平提供了坚实基础。我们的实验揭示了当前最先进LLMs在时间序列理解方面的优势与局限,明确了哪些特征是模型能有效掌握的,哪些方面存在不足。此外,我们还发现LLMs对数据格式、序列中被查询点的位置以及时间序列总长度等因素具有敏感性。

英文摘要:

Large Language Models (LLMs) offer the potential for automatic time series analysis and reporting, which is a critical task across many domains, spanning healthcare, finance, climate, energy, and many more. In this paper, we propose a framework for rigorously evaluating the capabilities of LLMs on time series understanding, encompassing both univariate and multivariate forms. We introduce a comprehensive taxonomy of time series features, a critical framework that delineates various characteristics inherent in time series data. Leveraging this taxonomy, we have systematically designed and synthesized a diverse dataset of time series, embodying the different outlined features, each accompanied by textual descriptions. This dataset acts as a solid foundation for assessing the proficiency of LLMs in comprehending time series. Our experiments shed light on the strengths and limitations of state-of-the-art LLMs in time series understanding, revealing which features these models readily comprehend effectively and where they falter. In addition, we uncover the sensitivity of LLMs to factors including the formatting of the data, the position of points queried within a series and the overall time series length.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑