学习绑定和生长神经结构
Learn to Bind and Grow Neural Structures
摘要:
任务增量学习涉及不断学习新任务的挑战性问题,而不会忘记过去的知识。许多方法通过在任务到达时扩展共享神经网络的结构来解决这个问题,但难以在不丢失过去知识的情况下实现最佳增长。我们提出了一个新的框架,学习绑定和增长,学习一个新的任务的神经架构增量,无论是通过绑定层的类似任务或扩展层之间更有可能发生冲突的任务。我们的方法的核心是一个新的,可解释的,参数化的共享,多任务架构空间,然后使计算全局最优架构使用贝叶斯优化。在持续学习基准测试上的实验表明,我们的框架与早期的基于扩展的方法进行了比较,并且能够灵活地计算多个最优解,同时兼顾性能和大小。
英文摘要:
Task-incremental learning involves the challenging problem of learning new tasks continually, without forgetting past knowledge. Many approaches address the problem by expanding the structure of a shared neural network as tasks arrive, but struggle to grow optimally, without losing past knowledge. We present a new framework, Learn to Bind and Grow, which learns a neural architecture for a new task incrementally, either by binding with layers of a similar task or by expanding layers which are more likely to conflict between tasks. Central to our approach is a novel, interpretable, parameterization of the shared, multi-task architecture space, which then enables computing globally optimal architectures using Bayesian optimization. Experiments on continual learning benchmarks show that our framework performs comparably with earlier expansion based approaches and is able to flexibly compute multiple optimal solutions with performance-size trade-offs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。