基于裂缝结构的粗到细非刚性多模态图像配准用于历史面板绘画
Coarse-to-Fine Non-rigid Multi-modal Image Registration for Historical Panel Paintings based on Crack Structures
AI总结：
本文提出了一种基于裂缝结构的粗到细非刚性多模态图像配准方法，利用稀疏关键点和薄板样条实现高精度的历史面板绘画图像配准。
摘要：
对历史面板绘画的艺术技术研究依赖于获取多模态图像数据，包括可见光摄影、红外反射摄影、紫外荧光摄影、X射线摄影和宏观摄影。为了进行全面分析，多模态图像需要逐像素对齐，这仍然常常手动完成。多模态图像配准可以减少这种繁琐的手动工作，速度显著提高，并能实现更高的精度。由于图像分辨率不同、图像尺寸巨大、非刚性变形以及模态依赖的图像内容，配准具有挑战性。因此，我们提出了一种基于稀疏关键点和薄板样条的粗到细非刚性多模态配准方法。历史绘画在颜料层上表现出细密的裂缝模式，称为裂纹，所有图像系统都能捕捉到这种裂纹，并且非常适合用于配准。在我们的一阶段非刚性配准方法中，我们使用卷积神经网络基于裂纹进行联合关键点检测和描述，并使用图神经网络以块为基础进行描述符匹配，并根据在局部区域的投影误差过滤匹配。对于粗到细配准，我们引入了一种新的多级关键点细化方法，以配准混合分辨率的图像，直到最高分辨率。我们创建了一个面板绘画的多模态数据集，具有大量的关键点注释，并且包含五个多模态领域和不同图像分辨率的大型测试集。消融研究证明了我们细化方法所有模块的有效性。我们提出的方法在与竞争的关键点和密集匹配方法以及细化方法相比，实现了最佳的配准结果。
英文摘要：
Art technological investigations of historical panel paintings rely on acquiring multi-modal image data, including visual light photography, infrared reflectography, ultraviolet fluorescence photography, x-radiography, and macro photography. For a comprehensive analysis, the multi-modal images require pixel-wise alignment, which is still often performed manually. Multi-modal image registration can reduce this laborious manual work, is substantially faster, and enables higher precision. Due to varying image resolutions, huge image sizes, non-rigid distortions, and modality-dependent image content, registration is challenging. Therefore, we propose a coarse-to-fine non-rigid multi-modal registration method efficiently relying on sparse keypoints and thin-plate-splines. Historical paintings exhibit a fine crack pattern, called craquelure, on the paint layer, which is captured by all image systems and is well-suited as a feature for registration. In our one-stage non-rigid registration approach, we employ a convolutional neural network for joint keypoint detection and description based on the craquelure and a graph neural network for descriptor matching in a patch-based manner, and filter matches based on homography reprojection errors in local areas. For coarse-to-fine registration, we introduce a novel multi-level keypoint refinement approach to register mixed-resolution images up to the highest resolution. We created a multi-modal dataset of panel paintings with a high number of keypoint annotations, and a large test set comprising five multi-modal domains and varying image resolutions. The ablation study demonstrates the effectiveness of all modules of our refinement method. Our proposed approaches achieve the best registration results compared to competing keypoint and dense matching methods and refinement methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
