从图库到手腕：在视频中实现逼真的3D手镯插入
From Gallery to Wrist: Realistic 3D Bracelet Insertion in Videos
AI总结：
针对视频3D物体插入中时间一致性与逼真光照难以兼顾的问题，提出融合3DGS初始渲染与2D扩散增强的着色驱动混合流水线，实现动态手腕场景下手镯的逼真一致插入。
摘要：
将3D物体插入视频是计算机图形学领域的一项长期挑战，可应用于增强现实、虚拟试穿和视频合成。要同时实现时间一致性与逼真光照效果仍然十分困难，尤其是在存在复杂物体运动、视角变化和光照变化的动态场景中。尽管2D扩散模型在生成照片级真实感的编辑效果方面展现出潜力，但它们往往难以维持帧间的时间连贯性。相反，传统3D渲染方法在空间和时间一致性上表现出色，但在实现照片级真实感光照方面存在不足。在本研究中，我们提出了一种混合式物体插入流水线，融合了两种范式的优势。具体而言，我们聚焦于将手镯插入动态手腕场景，利用3D Gaussian Splatting（3DGS，三维高斯泼溅）的高时间一致性进行初始渲染，再使用基于2D扩散的增强模型优化结果，以确保逼真的光照交互效果。我们的方法提出了一种着色驱动的流水线，分离物体的固有属性（反照率、着色、反射率），并对着色图像和sRGB图像均进行优化以实现照片级真实感。为维持时间连贯性，我们通过多帧加权调整来优化3DGS模型。这是首个将3D渲染与2D扩散协同应用于视频物体插入的方法，为逼真且一致的视频编辑提供了可靠的解决方案。项目页面：https://cjeen.github.io/BraceletPaper/
英文摘要：
Inserting 3D objects into videos is a longstanding challenge in computer graphics with applications in augmented reality, virtual try-on, and video composition. Achieving both temporal consistency, or realistic lighting remains difficult, particularly in dynamic scenarios with complex object motion, perspective changes, and varying illumination. While 2D diffusion models have shown promise for producing photorealistic edits, they often struggle with maintaining temporal coherence across frames. Conversely, traditional 3D rendering methods excel in spatial and temporal consistency but fall short in achieving photorealistic lighting. In this work, we propose a hybrid object insertion pipeline that combines the strengths of both paradigms. Specifically, we focus on inserting bracelets into dynamic wrist scenes, leveraging the high temporal consistency of 3D Gaussian Splatting (3DGS) for initial rendering and refining the results using a 2D diffusion-based enhancement model to ensure realistic lighting interactions. Our method introduces a shading-driven pipeline that separates intrinsic object properties (albedo, shading, reflectance) and refines both shading and sRGB images for photorealism. To maintain temporal coherence, we optimize the 3DGS model with multi-frame weighted adjustments. This is the first approach to synergize 3D rendering and 2D diffusion for video object insertion, offering a robust solution for realistic and consistent video editing. Project Page: https://cjeen.github.io/BraceletPaper/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
