Ferret：一种在变化内存约束下的高效在线持续学习框架
Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints
AI总结：
本文提出 Ferret 框架，通过细粒度流水线并行、迭代梯度补偿及自动化模型划分，在变化内存预算下提升 OCL 的实时准确率与效率。
摘要：
在高频数据流领域，在变化的内存约束下实现实时学习至关重要。本文提出 Ferret，这是一个综合框架，旨在提升在线持续学习（OCL）算法的在线准确率，同时动态适应不断变化的内存预算。Ferret 采用细粒度流水线并行策略，并结合迭代梯度补偿算法，确保以最小延迟无缝处理高频数据，并有效抵消并行训练中的陈旧梯度挑战。为适应变化的内存预算，其自动化模型划分与流水线规划可在不受内存限制的情况下优化性能。在 20 个基准和 5 种集成 OCL 算法上进行的大量实验表明，Ferret 具有显著效率：与竞争方法相比，在达到相同在线准确率时，其内存开销最高可降低 3.7×。此外，Ferret 在多种内存预算下始终优于这些方法，凸显了其卓越的适应性。这些发现使 Ferret 成为实时环境中高效、自适应 OCL 框架的首选解决方案。
英文摘要：
In the realm of high-frequency data streams, achieving real-time learning within varying memory constraints is paramount. This paper presents Ferret, a comprehensive framework designed to enhance online accuracy of Online Continual Learning (OCL) algorithms while dynamically adapting to varying memory budgets. Ferret employs a fine-grained pipeline parallelism strategy combined with an iterative gradient compensation algorithm, ensuring seamless handling of high-frequency data with minimal latency, and effectively counteracting the challenge of stale gradients in parallel training. To adapt to varying memory budgets, its automated model partitioning and pipeline planning optimizes performance regardless of memory limitations. Extensive experiments across 20 benchmarks and 5 integrated OCL algorithms show Ferret's remarkable efficiency, achieving up to 3.7$\times$ lower memory overhead to reach the same online accuracy compared to competing methods. Furthermore, Ferret consistently outperforms these methods across diverse memory budgets, underscoring its superior adaptability. These findings position Ferret as a premier solution for efficient and adaptive OCL framework in real-time environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
