arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.07019cs.CVcs.AIcs.CL
收藏

Musketeer:基于任务解释提示的多任务视觉语言模型联合训练

Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts

The Chinese University of Hong Kong(香港中文大学);AWS AI Labs(AWS AI实验室)

Zhaoyang Zhang, Yantao Shen, Kunyu Shi, Zhaowei Cai, Jun Fang, Siqi Deng, Hao Yang, Davide Modolo, Zhuowen Tu, Stefano Soatto

AI总结:

提出Musketeer,一种多任务视觉语言模型,通过任务解释提示(TEP)减少任务干扰,实现参数共享,性能媲美或超越单任务基线。

摘要:

我们提出了一种视觉语言模型,其参数在所有任务上联合训练,并在多个可能相互干扰的异构任务之间完全共享,从而得到一个我们命名为Musketeer的单一模型。跨异构任务的知识整合由一种名为任务解释提示(TEP)的新特性实现。凭借诸如任务输入/输出格式等丰富且结构化的信息,TEP减少了任务间的干扰,使模型能够专注于它们的共享结构。使用单一模型,Musketeer在多个任务上几乎一致地取得了与单任务训练的强基线相当或更好的结果。

英文摘要:

We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks which may interfere with each other, resulting in a single model which we named Musketeer. The integration of knowledge across heterogeneous tasks is enabled by a novel feature called Task Explanation Prompt (TEP). With rich and structured information such as task input/output format, TEP reduces interference among tasks, allowing the model to focus on their shared structure. With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑