深度时间序列模型中的记忆

中文摘要 AI 辅助

时间序列的深度学习已经通过一系列架构范式取得了进展，从循环网络和Transformer到结构化状态空间模型、检索增强预测器、基础模型以及使用工具的智能体。这些发展通常被孤立地研究，按架构或建模时代来组织。我们认为，它们反而可以通过一个共同的问题来审视：\u201c时间序列模型如何在其直接输入之外保留和访问信息？\u201d这个问题源于传统时间序列建模的一个根本限制：与预测相关的信息可能远远超出可行的输入窗口，而将历史压缩成固定大小的状态可能会丢弃以后可能有用的信息。我们将这一挑战表述为一个\u201c记忆\u201d问题，并将现有的时间序列方法组织在一个谱系上，从编码在参数和固定大小状态中的内部记忆，到可寻址、可检索且日益由智能体维护的外部记忆。然后，我们开发了一个统一的记忆机制分类法，并在一个共同框架下审视三类外部记忆，包括显式模块、检索增强和智能体存储，该框架涵盖保留什么、如何写入和访问以及如何持久化。一项跨领域分析将这些机制映射到时间序列任务，并识别了方法和评估中的空白。最后，我们概述了在构建能够随着时间环境演变而选择性保留、检索、修订和遗忘信息的记忆系统方面的开放问题。结果是一个将记忆作为时间序列建模的一等维度进行研究的框架，独立于底层骨干网络。

英文摘要

Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state-space models, retrieval-augmented predictors, foundation models, and tool-using agents. These developments are typically studied in isolation, organized by architecture or modeling era. We argue that they can instead be viewed through a common question of \emph{how does a time-series model retain and access information beyond its immediate input?} This question is motivated by a fundamental limitation of conventional time-series modeling: information relevant to a prediction may lie far beyond a feasible input window, while compressing history into a fixed-size state can discard information that may become useful later. We formulate this challenge as a \emph{memory} problem and organize existing time-series methods along a spectrum from internal memory, encoded in parameters and fixed-size states, to external memory that is addressable, retrievable, and increasingly maintained by agents. We then develop a unified taxonomy of memory mechanisms and review three classes of external memory, including explicit modules, retrieval augmentation, and agentic stores, under a common framework for what is retained, how it is written and accessed, and how it persists. A cross-cutting analysis maps these mechanisms to time-series tasks and identifies gaps in both methods and evaluation. We conclude by outlining open problems in building memory systems that can selectively retain, retrieve, revise, and forget information as temporal environments evolve. The result is a framework for studying memory as a first-class dimension of time series modeling, independent of the underlying backbone.

发表机构

• Deakin Applied AI Initiative, Deakin University（迪肯大学应用人工智能计划）

机构由 AI 辅助整理，请以论文原文为准。