GRILL：通过对齐文本与图像区域进行有根基的视觉-语言预训练
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions
AI总结：
本文提出GRILL模型，通过对象-文本对齐学习对象定位，使视觉-语言模型能在零样本或少样本设置下泛化到视觉问答、图像描述和定位等多种任务，并超越现有少样本方法。
摘要：
对未见任务的泛化是少样本学习者在多样化任务上取得更好零样本/少样本性能的重要能力。然而，这种对视觉-语言任务（包括定位和生成任务）的泛化尚未得到充分探索；现有的少样本视觉-语言模型难以处理涉及对象定位和多图像的任务，例如视觉常识推理或NLVR2。在本文中，我们提出了GRILL，即GRounded vIsion Language aLigning，这是一种新颖的视觉-语言模型，可以在没有或仅有极少训练实例的情况下泛化到多样化任务，包括视觉问答、图像描述和定位任务。具体而言，GRILL通过利用对象-文本对齐来学习对象定位和位置确定，这使其能够以零样本/少样本方式迁移到定位任务。我们在各种零样本/少样本视觉-语言任务上评估了我们的模型，并表明它持续超越了最先进的少样本方法。
英文摘要：
Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks. However, such generalization to vision-language tasks including grounding and generation tasks has been under-explored; existing few-shot VL models struggle to handle tasks that involve object grounding and multiple images such as visual commonsense reasoning or NLVR2. In this paper, we introduce GRILL, GRounded vIsion Language aLigning, a novel VL model that can be generalized to diverse tasks including visual question answering, captioning, and grounding tasks with no or very few training instances. Specifically, GRILL learns object grounding and localization by exploiting object-text alignments, which enables it to transfer to grounding tasks in a zero-/few-shot fashion. We evaluate our model on various zero-/few-shot VL tasks and show that it consistently surpasses the state-of-the-art few-shot methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
