通过上下文大表格模型实现流式持续学习
Bridging Streaming Continual Learning via In-Context Large Tabular Models
AI总结：
本文提出通过大规模上下文表格模型实现流式持续学习，结合流式学习与持续学习的核心原则，平衡塑性与稳定性，并控制内存大小。
摘要：
在流式场景中，模型必须持续学习，适应概念漂移而不遗忘之前获得的知识。然而，现有研究社区在解决这些挑战时往往是孤立的。持续学习（CL）侧重于长期保留和缓解灾难性遗忘，通常没有严格的实时约束。流式学习（SL）强调快速、高效地适应高频数据流，但通常忽视遗忘问题。最近的努力试图将这些范式结合，但尚未有明确的算法重叠。我们认为，大规模上下文表格模型（LTMs）为流式持续学习（SCL）提供了自然的桥梁。我们认为，无界的流式数据应在实时过程中被压缩成紧凑的草稿，供LTMs使用。这恢复了流式学习的经典动机，即通过固定大小的保证压缩大规模流式数据，同时与持续学习的经验回放需求相一致。为阐明这一桥梁，我们展示了SL和CL社区如何隐式采用分而治之的策略来管理塑性（在当前分布上表现良好）和稳定性（保留过去知识）之间的张力，同时施加最小的复杂性约束，这激励了多样化（避免存储中的冗余）和检索（在需要时重新优先过去的信息）。在此视角下，我们提出以LTMs为核心，围绕上下文学习的数据选择两个核心原则来构建SCL：（1）分布匹配，平衡塑性与稳定性；（2）分布压缩，通过多样化和检索机制控制内存大小。
英文摘要：
In streaming scenarios, models must learn continuously, adapting to concept drifts without erasing previously acquired knowledge. However, existing research communities address these challenges in isolation. Continual Learning (CL) focuses on long-term retention and mitigating catastrophic forgetting, often without strict real-time constraints. Stream Learning (SL) emphasizes rapid, efficient adaptation to high-frequency data streams, but typically neglects forgetting. Recent efforts have tried to combine these paradigms, yet no clear algorithmic overlap exists. We argue that large in-context tabular models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL). In our view, unbounded streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs. This recovers the classical SL motivation of compressing massive streams with fixed-size guarantees, while simultaneously aligning with the experience-replay desiderata of CL. To clarify this bridge, we show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity (performing well on the current distribution) and stability (retaining past knowledge), while also imposing a minimal complexity constraint that motivates diversification (avoiding redundancy in what is stored) and retrieval (re-prioritizing past information when needed). Within this perspective, we propose structuring SCL with LTMs around two core principles of data selection for in-context learning: (1) distribution matching, which balances plasticity and stability, and (2) distribution compression, which controls memory size through diversification and retrieval mechanisms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
