arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.07669cs.CVcs.CLcs.LG
收藏

视觉任务的统一序列接口

A Unified Sequence Interface for Vision Tasks

Google Research(谷歌研究院)

Ting Chen, Saurabh Saxena, Lala Li, Tsung-Yi Lin, David J. Fleet, Geoffrey Hinton

摘要:

虽然语言任务自然地表达在单个统一的建模框架中,即,生成标记序列,这在计算机视觉中不是这种情况。因此,对于不同的视觉任务,存在不同的架构和损失函数的扩散。在这项工作中,我们表明,一组不同的“核心”计算机视觉任务也可以统一,如果制定一个共享的像素序列接口。我们专注于四个任务,即目标检测,实例分割,关键点检测和图像描述,所有这些任务都具有不同类型的输出,例如,边界框或密集掩码。尽管如此,通过将每个任务的输出制定为具有统一接口的离散令牌序列,我们表明可以训练具有单一模型架构和所有这些任务的损失函数的神经网络,而无需特定于任务的定制。为了解决一个特定的任务,我们使用一个简短的提示作为任务描述,序列输出适应提示,因此它可以产生特定于任务的输出。我们表明,这样的模型可以实现有竞争力的性能相比,完善的特定任务的模型。

英文摘要:

While language tasks are naturally expressed in a single, unified, modeling framework, i.e., generating sequences of tokens, this has not been the case in computer vision. As a result, there is a proliferation of distinct architectures and loss functions for different vision tasks. In this work we show that a diverse set of "core" computer vision tasks can also be unified if formulated in terms of a shared pixel-to-sequence interface. We focus on four tasks, namely, object detection, instance segmentation, keypoint detection, and image captioning, all with diverse types of outputs, e.g., bounding boxes or dense masks. Despite that, by formulating the output of each task as a sequence of discrete tokens with a unified interface, we show that one can train a neural network with a single model architecture and loss function on all these tasks, with no task-specific customization. To solve a specific task, we use a short prompt as task description, and the sequence output adapts to the prompt so it can produce task-specific output. We show that such a model can achieve competitive performance compared to well-established task-specific models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑