释放大语言模型潜力：面向洪水灾害场景的零样本视觉问答
Unleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario
AI总结：
本文提出零样本洪水灾害损失评估VQA模型ZFDDA，构建含2,058张图像和22,422个问答对的FFD-IQA数据集，并利用思维链示例释放大语言模型潜力，显著提升复杂问题回答准确率。
摘要：
视觉问答（VQA）是一项基础且重要的人工智能任务，基于VQA的灾害场景理解是当前的研究热点。例如，我们可以通过VQA模型对灾害图像提出问题，其回答有助于判断是否有人或物受到灾害影响。然而，以往用于灾害损失评估的VQA模型存在一些不足，如候选答案空间有限、问题类型单调以及现有模型回答能力受限。本文提出了一种名为面向洪水灾害损失评估的零样本VQA模型（ZFDDA），它是一种无需预训练的灾害损失评估VQA模型。同时，以洪水灾害为主要研究对象，我们构建了一个自由式洪水灾害图像问答数据集（FFD-IQA）来评估我们的VQA模型。该新数据集将问题类型扩展为自由形式、多项选择和是非题三类。与此同时，我们扩充了此前数据集的规模，使其共包含2,058张图像和22,422个问题-元真值对。最重要的是，我们的模型利用精心设计的思维链（CoT）示例来释放大语言模型的潜力，使零样本VQA在灾害场景中展现出更优性能。实验结果表明，使用CoT提示后，回答复杂问题的准确率大幅提升。本研究为后续其他灾害场景下的VQA研究提供了研究基础。
英文摘要：
Visual question answering (VQA) is a fundamental and essential AI task, and VQA-based disaster scenario understanding is a hot research topic. For instance, we can ask questions about a disaster image by the VQA model and the answer can help identify whether anyone or anything is affected by the disaster. However, previous VQA models for disaster damage assessment have some shortcomings, such as limited candidate answer space, monotonous question types, and limited answering capability of existing models. In this paper, we propose a zero-shot VQA model named Zero-shot VQA for Flood Disaster Damage Assessment (ZFDDA). It is a VQA model for damage assessment without pre-training. Also, with flood disaster as the main research object, we build a Freestyle Flood Disaster Image Question Answering dataset (FFD-IQA) to evaluate our VQA model. This new dataset expands the question types to include free-form, multiple-choice, and yes-no questions. At the same time, we expand the size of the previous dataset to contain a total of 2,058 images and 22,422 question-meta ground truth pairs. Most importantly, our model uses well-designed chain of thought (CoT) demonstrations to unlock the potential of the large language model, allowing zero-shot VQA to show better performance in disaster scenarios. The experimental results show that the accuracy in answering complex questions is greatly improved with CoT prompts. Our study provides a research basis for subsequent research of VQA for other disaster scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
