arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.00097cs.LGcs.NEstat.ML
收藏

模块化通用重新参数化:跨不同领域的深度多任务学习

Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains

Cognizant(高知特);The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Elliot Meyerson, Risto Miikkulainen

摘要:

随着深度学习的应用越来越多样化,一个有趣的问题出现了:共同学习几个这样不同的任务是否可以解决一般问题?为了解决这个问题,本文将深度多任务学习扩展到任务架构之间没有明显重叠的环境。其思想是,任何一组(架构,任务)对可以分解成一组潜在相关的子问题,其共享是优化的一个有效的随机算法。该方法首先在经典的合成多任务学习基准中得到验证,然后应用于视觉,NLP和基因组学任务的不同架构之间的共享。它发现跨这些域的冗余,将它们编码为可共享的模块,并系统地组合这些模块以提高单个任务的性能。结果证实,在不同的领域和架构中共享学习到的功能确实是有益的,从而为未来解决一般问题建立了一个关键因素。

英文摘要:

As deep learning applications continue to become more diverse, an interesting question arises: Can general problem solving arise from jointly learning several such diverse tasks? To approach this question, deep multi-task learning is extended in this paper to the setting where there is no obvious overlap between task architectures. The idea is that any set of (architecture,task) pairs can be decomposed into a set of potentially related subproblems, whose sharing is optimized by an efficient stochastic algorithm. The approach is first validated in a classic synthetic multi-task learning benchmark, and then applied to sharing across disparate architectures for vision, NLP, and genomics tasks. It discovers regularities across these domains, encodes them into sharable modules, and combines these modules systematically to improve performance in the individual tasks. The results confirm that sharing learned functionality across diverse domains and architectures is indeed beneficial, thus establishing a key ingredient for general problem solving in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑