基于模型选择的深度弹性网络多任务学习
Deep Elastic Networks with Model Selection for Multi-Task Learning
摘要:
在这项工作中,我们考虑了多任务学习的实例动态网络模型选择问题。为此,我们提出了一种有效的方法来利用一个紧凑,但准确的模型在骨干架构的所有任务的每个实例。所提出的方法包括一个估计器和一个选择器。估计器是基于骨干架构和层次结构。它可以产生多个不同的网络模型的不同配置的层次结构。选择器从给定输入实例的候选模型池中动态选择模型。选择器是一个由几层组成的相对较小的网络,当给定任务的输入实例时,它估计候选模型的概率分布。估计器和选择器都在统一的学习框架中结合基于采样的学习策略进行联合训练,而无需额外的计算步骤。我们证明了几个图像分类任务相比,现有的方法进行模型选择或学习多个任务所提出的方法。实验结果表明,与其他竞争对手相比,我们的方法不仅具有出色的性能,而且具有执行多任务实例模型选择的通用性。
英文摘要:
In this work, we consider the problem of instance-wise dynamic network model selection for multi-task learning. To this end, we propose an efficient approach to exploit a compact but accurate model in a backbone architecture for each instance of all tasks. The proposed method consists of an estimator and a selector. The estimator is based on a backbone architecture and structured hierarchically. It can produce multiple different network models of different configurations in a hierarchical structure. The selector chooses a model dynamically from a pool of candidate models given an input instance. The selector is a relatively small-size network consisting of a few layers, which estimates a probability distribution over the candidate models when an input instance of a task is given. Both estimator and selector are jointly trained in a unified learning framework in conjunction with a sampling-based learning strategy, without additional computation steps. We demonstrate the proposed approach for several image classification tasks compared to existing approaches performing model selection or learning multiple tasks. Experimental results show that our approach gives not only outstanding performance compared to other competitors but also the versatility to perform instance-wise model selection for multiple tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。