GeoFix:用于4D一致性生成的几何修复
GeoFix: Geometry Fixing for 4D Consistent Generation
AI总结:
GeoFix是一个统一框架,通过校准的单一几何奖励提升4D生成的几何一致性与视觉质量,在WorldScore 3D一致性和VBench质量得分上分别实现39%和4%的提升。
摘要:
视频扩散模型日益成为4D生成的先验,但生成的视频常缺乏几何一致性。现有方法通过设计几何奖励进行后训练来解决该问题,但这会降低纹理和精细细节,需要辅助外观奖励及精细的奖励平衡。为应对这一挑战,我们提出GeoFix,这是一个统一框架,可通过单一几何奖励提升几何一致性与视觉质量,适用于训练和推理时的奖励对齐。我们的奖励测量静态场景区域的跨视图深度一致性,针对结构不一致性,不直接约束外观或惩罚有效物体运动。关键是,即使在真实视频和4D数据集中,深度一致性也随相机视场重叠度增加而提升,这会导致相机移动较小时被误认为几何效果更好。我们通过针对依赖重叠度的趋势校准奖励来解决该效应。GeoFix在WorldScore 3D一致性上实现39%的提升,在VBench质量得分上实现4%的增长。
英文摘要:
Video diffusion models increasingly serve as priors for 4D generation, but the generated videos often lack geometric consistency. Recent approaches address this through post-training by designing geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves geometric consistency and visual quality with a single geometric reward, applicable to both training and inference-time reward alignment. Our reward measures cross-view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field-of-view overlap even in real videos and 4D datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap-dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。