arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.09884cs.CVcs.IR
收藏

基于图像的半监督叙事提取:以历史摄影记录为例的研究

Semi-Supervised Image-Based Narrative Extraction: A Case Study with Historical Photographic Records

Virginia Tech(弗吉尼亚理工大学);Universidad Católica del Norte(北方天主教大学)

Fausto German, Brian Keith, Mauricio Matus, Diego Urrutia, Claudio Meneses

AI总结:

本研究提出适配图像的半监督叙事地图算法,从历史摄影记录中提取叙事,在ROGER数据集上验证了其在长时序叙事提取上优于随机采样,为视觉文化遗产分析提供新工具。

摘要:

本文提出一种半监督方法,通过适配叙事地图(narrative maps)算法从历史摄影记录中提取叙事。我们将原本基于文本的无监督方法扩展至适配图像数据,利用深度学习技术进行视觉特征提取与相似度计算。我们的方法应用于ROGER数据集——这是1928年玻利维亚萨卡姆巴亚探险中由Robert Gerstmann拍摄的照片合集。我们将算法提取的视觉叙事与不同长度(5至30张图像)的专家整理时间线进行对比,以评估方法的有效性。具体而言,我们使用动态时间规整(Dynamic Time Warping,DTW)算法将提取的叙事与专家整理的基准进行匹配。此外,我们邀请了该领域的一位专家对生成叙事的代表性样例进行定性评估。研究结果表明,对于更长的时间线(10张及以上图像,p < 0.05),叙事地图方法的表现整体优于随机采样,专家评估也证实了提取叙事的历史准确性与连贯性。本研究为视觉文化遗产计算分析领域做出贡献,为历史学家、档案管理员和数字人文学者探索和理解大规模图像合集提供了新工具。该方法从视觉数据中生成有意义叙事的能力,为通过摄影证据研究和解读历史事件开辟了新的可能性。

英文摘要:

This paper presents a semi-supervised approach to extracting narratives from historical photographic records using an adaptation of the narrative maps algorithm. We extend the original unsupervised text-based method to work with image data, leveraging deep learning techniques for visual feature extraction and similarity computation. Our method is applied to the ROGER dataset, a collection of photographs from the 1928 Sacambaya Expedition in Bolivia captured by Robert Gerstmann. We compare our algorithmically extracted visual narratives with expert-curated timelines of varying lengths (5 to 30 images) to evaluate the effectiveness of our approach. In particular, we use the Dynamic Time Warping (DTW) algorithm to match the extracted narratives with the expert-curated baseline. In addition, we asked an expert on the topic to qualitatively evaluate a representative example of the resulting narratives. Our findings show that the narrative maps approach generally outperforms random sampling for longer timelines (10+ images, p < 0.05), with expert evaluation confirming the historical accuracy and coherence of the extracted narratives. This research contributes to the field of computational analysis of visual cultural heritage, offering new tools for historians, archivists, and digital humanities scholars to explore and understand large-scale image collections. The method's ability to generate meaningful narratives from visual data opens up new possibilities for the study and interpretation of historical events through photographic evidence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑