SHARP: 基于睡眠的分层加速重放用于长程非平稳时间模式识别
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
AI总结：
提出SHARP框架，通过将时间学习分解为记忆模块和模式识别模块，并引入离线睡眠阶段加速重放时间结构记忆，实现长程非平稳序列模式的高效学习。
摘要：
学习长程非平稳时间模式仍然是现代序列模型的核心挑战，特别是在严格的流式设置中。在这些设置中，数据按顺序到达，必须单次处理，不能同时回顾过去的观测。标准架构，包括循环神经网络和变换器，受到截断时间反向传播或显式输入窗口长度的限制，无法进行长程信用分配。为了解决这些限制，我们提出了SHARP（基于睡眠的分层加速重放），一个将时间学习分解为两个互补组件的框架：一个累积过去输入的结构化历史的记忆模块，以及一个在该记忆上操作的模式识别模块。这种分离通过消除跨多步时间反向传播进行长程信用分配的需求，实现了对非平稳动态的资源高效和计算高效适应。受啮齿动物在慢波睡眠期间观察到的加速重放启发，SHARP引入了离线（睡眠）阶段，其中时间结构的记忆痕迹以加速形式重放并整合到更高层次的记忆表示中，从而改善长程上下文保留。通过受控模拟和消融研究，我们表征了所提出框架的关键属性。在text8和PG-19等基准数据集上，我们证明SHARP通过保留先前见过数据的下一个令牌预测性能，同时继续从当前流中学习并泛化到未来未见数据，改进了循环基线。这些增益得益于其分层结构，该结构以线性时间计算成本实现了指数级增长的有效时间上下文。
英文摘要：
Learning long-range non-stationary temporal patterns remains a core challenge for modern sequence models, particularly in strict streaming settings. In these settings, data arrive sequentially and must be processed in a single pass without simultaneously revisiting past observations. Standard architectures, including recurrent neural networks and transformers, are constrained by either truncated backpropagation through time horizon or explicit input window length for long range credit assignment. To address these limitations, we propose SHARP (Sleep-based Hierarchical Accelerated Replay), a framework that decomposes temporal learning into two complementary components: a memory module that accumulates a structured history of past inputs, and a pattern-recognition module that operates over this memory. This separation enables resource- and compute-efficient adaptation to non-stationary dynamics by eliminating the need for backpropagation through time across many steps for long-range credit assignment. Inspired by the accelerated replay observed in rodents during slow-wave sleep, SHARP incorporates offline (sleep) phases in which temporally structured memory traces are replayed in an accelerated form and integrated into higher-level memory representations, improving long-range context retention. Through controlled simulations and ablation studies, we characterize the key properties of the proposed framework. In benchmark datasets such as text8 and PG-19, we demonstrate that SHARP improves over recurrent baselines by retaining next-token predictive performance on previously seen data while continuing to learn from the current stream and generalizing to future unseen data. These gains are enabled by its hierarchical structure, which yields an exponentially increasing effective temporal context with only linear-time computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
