VD-BERT：与BERT的统一愿景和对话Transformer
VD-BERT: A Unified Vision and Dialog Transformer with BERT
摘要：
视觉对话是一个具有挑战性的视觉语言任务，其中对话代理需要通过对图像内容和对话历史的推理来回答一系列问题。先前的工作主要集中在各种注意力机制，以模拟这种复杂的相互作用。相比之下，在这项工作中，我们提出了VD-BERT，一个简单而有效的统一视觉对话Transformer框架，它利用预先训练的BERT语言模型进行视觉对话任务。该模型是统一的，因为（1）它使用单流Transformer编码器捕获图像和多轮对话之间的所有交互，并且（2）它通过相同的架构无缝地支持答案排名和答案生成。更重要的是，我们通过基于视觉的训练来适应BERT，以有效地融合视觉和对话内容。在不需要外部视觉语言数据预训练的情况下，我们的模型产生了最新的技术水平，在视觉对话排行榜上的单模型和集成设置（74.54和75.35 NDCG得分）中均获得了最高位置。我们的代码和预训练模型在https://github.com/salesforce/VD-BERT上发布。
英文摘要：
Visual dialog is a challenging vision-language task, where a dialog agent needs to answer a series of questions through reasoning on the image content and dialog history. Prior work has mostly focused on various attention mechanisms to model such intricate interactions. By contrast, in this work, we propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer that leverages the pretrained BERT language models for Visual Dialog tasks. The model is unified in that (1) it captures all the interactions between the image and the multi-turn dialog using a single-stream Transformer encoder, and (2) it supports both answer ranking and answer generation seamlessly through the same architecture. More crucially, we adapt BERT for the effective fusion of vision and dialog contents via visually grounded training. Without the need of pretraining on external vision-language data, our model yields new state of the art, achieving the top position in both single-model and ensemble settings (74.54 and 75.35 NDCG scores) on the visual dialog leaderboard. Our code and pretrained models are released at https://github.com/salesforce/VD-BERT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
