一种高效的自动地图叙事系统——以历史地图为例
An Efficient System for Automatic Map Storytelling -- A Case Study on Historical Maps
AI总结：
本文提出一种轻量级历史地图自动叙事系统，通过微调CLIP生成地图描述并结合GPT-3.5丰富叙事，利用决策树架构限定地图类型，实现对文本改动的不变性，且易于扩展。
摘要：
历史地图提供了关于过去的宝贵信息和知识。然而，由于它们通常具有非标准投影、手绘风格和艺术元素，非专业人士难以识别和解读它们。尽管现有的图像描述方法在自然图像上取得了显著成功，但它们在地图上的表现并不理想，因为地图在其预训练过程中代表性不足。尽管GPT-4最近在文本识别和地图描述方面取得了进展，但它对地图的理解仍然有限，因为当地图中的文本（例如标题和图例）缺失或不准确时，其性能会下降。此外，使用用户自己的数据集对模型进行微调效率低下，甚至不切实际。为了解决这些问题，我们提出了一种新颖且轻量级的地图描述对应方案。具体来说，我们微调了最先进的视觉-语言模型CLIP，以生成与历史地图相关的描述，并使用GPT-3.5丰富这些描述，以讲述关于给定地图的地点、内容、时间和原因的简短故事。我们提出了一种新颖的决策树架构，仅生成与指定地图类型相关的描述。我们的系统对地图中的文本改动表现出不变性。该系统可以轻松适应并扩展到其他地图类型，并扩展为更大的地图描述系统。代码已在 https://github.com/claudaff/automatic-map-storytelling 开源。
英文摘要：
Historical maps provide valuable information and knowledge about the past. However, as they often feature non-standard projections, hand-drawn styles, and artistic elements, it is challenging for non-experts to identify and interpret them. While existing image captioning methods have achieved remarkable success on natural images, their performance on maps is suboptimal as maps are underrepresented in their pre-training process. Despite the recent advance of GPT-4 in text recognition and map captioning, it still has a limited understanding of maps, as its performance wanes when texts (e.g., titles and legends) in maps are missing or inaccurate. Besides, it is inefficient or even impractical to fine-tune the model with users' own datasets. To address these problems, we propose a novel and lightweight map-captioning counterpart. Specifically, we fine-tune the state-of-the-art vision-language model CLIP to generate captions relevant to historical maps and enrich the captions with GPT-3.5 to tell a brief story regarding where, what, when and why of a given map. We propose a novel decision tree architecture to only generate captions relevant to the specified map type. Our system shows invariance to text alterations in maps. The system can be easily adapted and extended to other map types and scaled to a larger map captioning system. The code is open-sourced at https://github.com/claudaff/automatic-map-storytelling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
