用于模块化多任务ML系统的多路径代理
Multipath agents for modular multitask ML systems
摘要:
一个标准的ML模型通常是由一个单一的方法,指定方面,如架构,初始化,训练数据和超参数配置生成。所提出的工作介绍了一种新的方法,允许定义多种方法作为不同的代理。代理可以协作和竞争,为给定的任务生成和改进ML模型。所提出的方法证明了一个动态模块化的多任务ML系统的生成和扩展,解决了一百多个图像分类任务。不同的代理可以通过重用由竞争代理引入系统的模块来竞争为任务产生最佳性能模型。所提出的工作集中在研究的代理能够:1)重用并发代理生成的模块,2)并行激活多个模块在冻结状态下通过连接它们与可训练的模块,3)条件的激活混合物的每个数据样本通过使用可训练的路由器模块。我们证明了这种简单的每样本并行路由方法可以通过训练一部分激活参数来提高组合解决方案的质量。
英文摘要:
A standard ML model is commonly generated by a single method that specifies aspects such as architecture, initialization, training data and hyperparameters configuration. The presented work introduces a novel methodology allowing to define multiple methods as distinct agents. Agents can collaborate and compete to generate and improve ML models for a given tasks. The proposed methodology is demonstrated with the generation and extension of a dynamic modular multitask ML system solving more than one hundred image classification tasks. Diverse agents can compete to produce the best performing model for a task by reusing the modules introduced to the system by competing agents. The presented work focuses on the study of agents capable of: 1) reusing the modules generated by concurrent agents, 2) activating in parallel multiple modules in a frozen state by connecting them with trainable modules, 3) condition the activation mixture on each data sample by using a trainable router module. We demonstrate that this simple per-sample parallel routing method can boost the quality of the combined solutions by training a fraction of the activated parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。