UViM:一种基于学习引导码的视觉统一建模方法
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes
摘要:
我们介绍UViM,一个统一的方法,能够建模范围广泛的计算机视觉任务。与以前的模型相比,UViM对所有任务都具有相同的功能形式;它不需要对特定任务进行修改,而这些修改需要大量的人类专业知识。该方法涉及两个组件:(I)基础模型(前馈),其被训练以直接预测原始视觉输出,由学习的离散代码引导;以及(II)语言模型(自回归),其被训练以生成引导代码。这些组件相互补充:语言模型非常适合于对结构化的相互依赖数据进行建模,而基础模型在处理高维输出方面非常有效。我们证明了UViM在三个不同的和具有挑战性的视觉任务的有效性:全景分割,深度预测和图像着色,在那里我们实现了竞争力和接近国家的最先进的结果。我们的实验结果表明,UViM是一个很有前途的候选人的统一建模方法在计算机视觉。
英文摘要:
We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks. In contrast to previous models, UViM has the same functional form for all tasks; it requires no task-specific modifications which require extensive human expertise. The approach involves two components: (I) a base model (feed-forward) which is trained to directly predict raw vision outputs, guided by a learned discrete code and (II) a language model (autoregressive) that is trained to generate the guiding code. These components complement each other: the language model is well-suited to modeling structured interdependent data, while the base model is efficient at dealing with high-dimensional outputs. We demonstrate the effectiveness of UViM on three diverse and challenging vision tasks: panoptic segmentation, depth prediction and image colorization, where we achieve competitive and near state-of-the-art results. Our experimental results suggest that UViM is a promising candidate for a unified modeling approach in computer vision.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。