Thoth：中期训练使LLM具备时间序列理解能力
Thoth: Mid-Training Bridges LLMs to Time Series Understanding
AI总结：
Thoth通过中期训练和Book-of-Thoth语料库，使LLM具备时间序列理解能力，并在多个基准测试中表现优异。
摘要：
大型语言模型（LLMs）在通用推理任务中表现出色，但仍然难以理解和推理时间序列数据，这限制了其在依赖时间动态的决策场景中的有效性。本文提出Thoth，这是首个具有通用时间序列理解能力的中期训练LLM家族。作为关键的中间阶段，中期训练实现了时间序列与自然语言之间的任务和领域无关的对齐，为此我们构建了Book-of-Thoth，一个高质量的时间序列中心中期训练语料库。Book-of-Thoth支持时间序列到文本和文本到时间序列的生成，使LLM具备对时间模式的基础理解。为进一步评估高级推理能力，我们还提出了KnoTS，一个新的知识密集型时间序列理解基准，旨在联合推理时间模式和领域知识。广泛实验表明，使用Book-of-Thoth进行中期训练使Thoth在一系列时间序列问答基准测试中显著优于其基础模型和高级LLM。此外，Thoth在数据稀缺情况下微调时表现出卓越的能力，证明了中期训练在时间序列理解中的有效性。代码可在：https://github.com/thuml/Thoth获取。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable success in general-purpose reasoning. However, they still struggle to understand and reason about time series data, which limits their effectiveness in decision-making scenarios that depend on temporal dynamics. In this paper, we propose Thoth, the first family of mid-trained LLMs with general-purpose time series understanding capabilities. As a pivotal intermediate stage, mid-training achieves task- and domain-agnostic alignment between time series and natural language, for which we construct Book-of-Thoth, a high-quality, time-series-centric mid-training corpus. Book-of-Thoth enables both time-series-to-text and text-to-time-series generation, equipping LLMs with a foundational grasp of temporal patterns. To better evaluate advanced reasoning capabilities, we further present KnoTS, a novel benchmark of knowledge-intensive time series understanding, designed for joint reasoning over temporal patterns and domain knowledge. Extensive experiments demonstrate that mid-training with Book-of-Thoth enables Thoth to significantly outperform its base model and advanced LLMs across a range of time series question answering benchmarks. Moreover, Thoth exhibits superior capabilities when fine-tuned under data scarcity, underscoring the effectiveness of mid-training for time series understanding. Code is available at: https://github.com/thuml/Thoth.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
