Simple Lifelong Learning Machines（简单终身学习机器）
Simple Lifelong Learning Machines
AI总结：
针对终身学习中现有方法多仅追求避免遗忘、目标偏低的问题，提出表示集成的简单方法，在多类基准场景中同时实现前向和后向迁移，且可灵活适配有无计算预算的情况。
摘要：
在终身学习中，数据不仅被用于提升当前任务的性能，还用于提升过去和未来（未遇到的）任务的性能。虽然典型的迁移学习算法可以提升未来任务的性能，但它们在学习新任务时，先前任务的性能会下降（这被称为遗忘）。近期许多针对持续学习或终身学习的方法都试图在学习新任务时维持旧任务的性能。但努力避免遗忘将目标设定得不必要地低。终身学习的目标应当是利用数据同时提升未来任务（前向迁移）和过去任务（后向迁移）的性能。在本文中，我们展示了一种简单的方法——表示集成——在多种模拟和基准数据场景中都能同时实现前向和后向迁移，涵盖表格数据、视觉（CIFAR-100、5-dataset、Split Mini-Imagenet和Food1k）以及语音（口语数字）场景；而各类参考算法通常无法实现前向迁移或后向迁移，甚至两者都无法实现。此外，我们提出的方法可以在有或没有计算预算的情况下灵活运行。
英文摘要：
In lifelong learning, data are used to improve performance not only on the present task, but also on past and future (unencountered) tasks. While typical transfer learning algorithms can improve performance on future tasks, their performance on prior tasks degrades upon learning new tasks (called forgetting). Many recent approaches for continual or lifelong learning have attempted to maintain performance on old tasks given new tasks. But striving to avoid forgetting sets the goal unnecessarily low. The goal of lifelong learning should be to use data to improve performance on both future tasks (forward transfer) and past tasks (backward transfer). In this paper, we show that a simple approach -- representation ensembling -- demonstrates both forward and backward transfer in a variety of simulated and benchmark data scenarios, including tabular, vision (CIFAR-100, 5-dataset, Split Mini-Imagenet, and Food1k), and speech (spoken digit), in contrast to various reference algorithms, which typically failed to transfer either forward or backward, or both. Moreover, our proposed approach can flexibly operate with or without a computational budget.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
