arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2509.01964cs.CVcs.AI

面向图像修复的带语义对齐的2D高斯泼溅(Gaussian Splatting)

2D Gaussian Splatting with Semantic Alignment for Image Inpainting

Hongyu Li, Chaofeng Chen, Xiaoming Li, Guangming Lu

AI总结:

本文提出首个基于2D高斯泼溅的图像修复框架,结合分块光栅化提升效率,引入预训练DINO特征保障全局语义一致性,在标准基准上取得了兼具量化指标与感知质量的竞争力性能,开辟了高斯泼溅在2D图像处理的新方向。

摘要:

高斯泼溅(Gaussian Splatting, GS)是近期一种将离散点转换为连续空间表示的技术,已在3D场景建模和2D图像超分辨率任务中展现出良好效果。本文探索了其在图像修复领域尚未被挖掘的潜力,该任务同时要求局部连贯的像素合成与全局一致的语义恢复。我们提出了首个基于2D高斯泼溅的图像修复框架,该框架将不完整图像编码为2D高斯泼溅系数的连续场,并通过可微光栅化过程重建最终图像。GS的连续渲染范式从本质上提升了修复结果的像素级连贯性。为提高效率与可扩展性,我们引入了分块光栅化策略,可降低内存开销并加速推理。为实现全局语义一致性,我们融合了预训练DINO模型的特征。我们观察到,DINO的全局特征天然对小面积缺失区域具有鲁棒性,且可有效适配以引导大掩码场景下的语义对齐,确保修复内容与周围场景保持上下文一致性。在标准基准上的大量实验表明,我们的方法在量化指标与感知质量上均取得了具有竞争力的性能,为高斯泼溅应用于2D图像处理开辟了新方向。

英文摘要:

Gaussian Splatting (GS), a recent technique for converting discrete points into continuous spatial representations, has shown promising results in 3D scene modeling and 2D image super-resolution. In this paper, we explore its untapped potential for image inpainting, which demands both locally coherent pixel synthesis and globally consistent semantic restoration. We propose the first image inpainting framework based on 2D Gaussian Splatting, which encodes incomplete images into a continuous field of 2D Gaussian splat coefficients and reconstructs the final image via a differentiable rasterization process. The continuous rendering paradigm of GS inherently promotes pixel-level coherence in the inpainted results. To improve efficiency and scalability, we introduce a patch-wise rasterization strategy that reduces memory overhead and accelerates inference. For global semantic consistency, we incorporate features from a pretrained DINO model. We observe that DINO's global features are naturally robust to small missing regions and can be effectively adapted to guide semantic alignment in large-mask scenarios, ensuring that the inpainted content remains contextually consistent with the surrounding scene. Extensive experiments on standard benchmarks demonstrate that our method achieves competitive performance in both quantitative metrics and perceptual quality, establishing a new direction for applying Gaussian Splatting to 2D image processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑