arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11147cs.MMcs.CVcs.LG
收藏

博物馆视频下的资源与监管约束下的目录引导多模态归因

Catalogue-Grounded Multimodal Attribution for Museum Video under Resource and Regulatory Constraints

University of Surrey(萨里大学)

Minsak Nanang, Adrian Hilton, Armin Mustafa

AI总结:

本文提出一种目录引导的多模态归因方法,用于博物馆视频内容的自动化元数据生成,以提高档案馆发现性并满足资源和监管要求。

摘要:

博物馆和画廊的音频视频档案正在快速增长,但大部分内容仍因缺乏一致且可搜索的元数据而无法有效利用。现有的存档方法需要大量人工工作。我们通过自动化工作流程中最耗费人力的部分来解决这个问题:为展厅视频生成目录风格的元数据,基于现有的收藏数据库。具体而言,我们提出了一种使用可部署的视频语言模型的目录引导多模态归因方法,用于博物馆音频视频内容。我们设计了一个多阶段流程,包括(i)总结视频中的艺术品,(ii)生成目录风格的描述和类型标签,以及(iii)通过保守的相似性匹配尝试归因标题和艺术家。在绘画目录上的早期部署表明,该框架可以在尊重资源限制、数据主权和新兴法规的情况下提高音频视频档案的可发现性,提供了一个适用于其他高风险领域的可转移模板。

英文摘要:

Museum audiovisual archives are important records of visual culture, yet in-gallery video remains difficult to search because it lacks catalogue-compatible metadata. We propose CaGMulA, catalogue-grounded multimodal attribution for museum video: a locally deployable pipeline that combines video-language models, structured catalogue retrieval, and abstention-aware decision policies. The system summarises visible artworks, generates catalogue-style descriptions, and attributes title and artist only when evidence is sufficient. We evaluate VideoLLaMA2 and Qwen2-VL across four model configurations and four attribution modes on 18 in-gallery videos. Open-ended generation fails under conservative conditions, producing 0\% coverage across models. By contrast, closed-set catalogue retrieval reaches 100\% coverage with 62\% accepted title--artist correctness, while a verifier-gated hybrid mode reduces coverage to 50\% but improves accepted correctness to 78\%. The results show that reliable museum-video attribution is driven less by unrestricted generation than by catalogue grounding, visual retrieval, and selective abstention for curatorial review.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑