超越像素的问题：在遥感中整合常识知识进行视觉问题生成
Questions beyond Pixels: Integrating Commonsense Knowledge in Visual Question Generation for Remote Sensing
AI总结：
本文提出KRSVQG模型，通过整合常识知识和图像内容，生成丰富的问题以提升遥感图像的语义理解和应用。
摘要：
随着遥感图像档案的快速发展，通过图像提问来获取特定信息或进行语义图像检索已成为一种有效的方法。然而，当前自动生成的问题往往过于简单且基于模板，这阻碍了在真实应用场景中部署问答或视觉对话系统。为了使问题更加丰富和多样化，结合图像内容和常识知识，我们提出了一种知识感知的遥感视觉问题生成模型（KRSVQG）。该模型通过整合外部知识源的相关知识三元组来拓宽问题内容，同时利用图像描述作为中介表示来将问题与相应图像关联起来。此外，KRSVQG采用了一种视觉-语言预训练和微调策略，使模型能够适应低数据环境。为了评估所提出的KRSVQG模型，我们构建了两个知识感知的遥感视觉问题生成数据集：NWPU-300数据集和TextRS-300数据集。评估包括指标和人工评估，证明KRSVQG优于现有方法，并产生基于图像和领域知识的丰富问题。作为视觉-语言研究中的关键实践，知识感知的视觉问题生成推动了对图像内容的理解超越像素，促进了具有视觉基础的人类常识的知识增强视觉-语言系统的发展。
英文摘要：
With the rapid development of remote sensing image archives, asking questions about images has become an effective way of gathering specific information or performing semantic image retrieval. However, current automatically generated questions tend to be simplistic and template-based, which hinders the deployment of question answering or visual dialogue systems for real-world applications. To enrich and diversify the questions with both image content and commonsense knowledge, we propose a Knowledge-aware Remote Sensing Visual Question Generation model (KRSVQG). The proposed model incorporates related knowledge triplets from external knowledge sources to broaden the question content, while employing image captioning as an intermediary representation to ground questions to the corresponding images. Moreover, KRSVQG utilizes a vision-language pre-training and fine-tuning strategy, enabling the model's adaptation to low data regimes. To evaluate the proposed KRSVQG model, we construct two knowledge-aware remote sensing visual question generation datasets: the NWPU-300 dataset and the TextRS-300 dataset. Evaluations, including metrics and human assessment, demonstrate that KRSVQG outperforms existing methods and leads to rich questions, grounded in both image and domain knowledge. As a key practice in vision-language research, knowledge-aware visual question generation advances the understanding of image content beyond pixels, facilitating the development of knowledge-enriched vision-language systems with vision-grounded human commonsense.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
