arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.06533cs.LGcs.AIcs.CEcs.CLstat.ME
收藏

LETS-C:利用文本嵌入进行时间序列分类

LETS-C: Leveraging Text Embedding for Time Series Classification

J.P. Morgan AI Research(摩根大通人工智能研究)

Rachneet Kaur, Zhen Zeng, Tucker Balch, Manuela Veloso

AI总结:

本文提出LETS-C方法,利用文本嵌入模型编码时间序列数据,并结合CNN和MLP组成的简单分类头进行分类,在准确率超越SOTA的同时大幅减少模型参数。

摘要:

语言建模的最新进展在应用于时间序列数据时展现出了令人鼓舞的结果。特别是,针对时间序列分类任务微调预训练大型语言模型(LLM)在标准基准测试上取得了最先进(SOTA)的性能。然而,这些基于LLM的模型由于模型规模庞大存在一个显著缺点,可训练参数达数百万。在本文中,我们提出了一种在时间序列领域利用语言建模成功的替代方法。我们不微调LLM,而是利用文本嵌入模型对时间序列进行嵌入,然后将嵌入与由卷积神经网络(CNN)和多层感知机(MLP)组成的简单分类头配对。我们在一个成熟的时间序列分类基准上进行了广泛实验。我们证明LETS-C不仅在分类准确率上超越了当前SOTA,还提供了一个轻量级解决方案,与SOTA模型相比平均仅使用14.5%的可训练参数。我们的研究结果表明,利用文本嵌入模型对时间序列数据进行编码,结合简单而有效的分类头,为实现高性能时间序列分类同时保持轻量级模型架构提供了一个有前景的方向。

英文摘要:

Recent advancements in language modeling have shown promising results when applied to time series data. In particular, fine-tuning pre-trained large language models (LLMs) for time series classification tasks has achieved state-of-the-art (SOTA) performance on standard benchmarks. However, these LLM-based models have a significant drawback due to the large model size, with the number of trainable parameters in the millions. In this paper, we propose an alternative approach to leveraging the success of language modeling in the time series domain. Instead of fine-tuning LLMs, we utilize a text embedding model to embed time series and then pair the embeddings with a simple classification head composed of convolutional neural networks (CNN) and multilayer perceptron (MLP). We conducted extensive experiments on a well-established time series classification benchmark. We demonstrated LETS-C not only outperforms the current SOTA in classification accuracy but also offers a lightweight solution, using only 14.5% of the trainable parameters on average compared to the SOTA model. Our findings suggest that leveraging text embedding models to encode time series data, combined with a simple yet effective classification head, offers a promising direction for achieving high-performance time series classification while maintaining a lightweight model architecture.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑