PatchFusion：面向高分辨率单目度量深度估计的端到端分块框架
PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation
AI总结：
PatchFusion提出端到端分块框架，通过逐块融合、全局到局部模块和一致性感知训练推理，解决高分辨率单目深度估计中的误差传播与细节丢失问题，并在多个基准上显著降低RMSE。
摘要：
单图像深度估计是计算机视觉和生成建模中的基础任务。然而，当前主流的深度估计模型难以适应当今消费级相机和设备中日益普遍的高分辨率输入。现有的高分辨率策略虽然展现出潜力，但往往面临从误差传播到高频细节丢失等一系列局限。我们提出PatchFusion，这是一种新颖的基于分块的框架，包含三个关键组件以改进当前最优水平：（1）一个逐块融合网络，通过高层特征引导，将全局一致的粗略预测与更精细但不一致的分块预测进行融合；（2）一个全局到局部（G2L）模块，为融合网络补充关键上下文，从而不再需要分块选择启发式方法；（3）一种一致性感知训练（CAT）与推理（CAI）方法，强调分块重叠一致性，从而消除了后处理的必要性。在UnrealStereo4K、MVS-Synth和Middleburry 2014上的实验表明，我们的框架能够生成具有精细细节的高分辨率深度图。PatchFusion独立于用于深度估计的基础模型。值得注意的是，我们构建在SOTA ZoeDepth之上的框架在UnrealStereo4K和MVS-Synth上分别将均方根误差（RMSE）改善了17.3%和29.4%。
英文摘要：
Single image depth estimation is a foundational task in computer vision and generative modeling. However, prevailing depth estimation models grapple with accommodating the increasing resolutions commonplace in today's consumer cameras and devices. Existing high-resolution strategies show promise, but they often face limitations, ranging from error propagation to the loss of high-frequency details. We present PatchFusion, a novel tile-based framework with three key components to improve the current state of the art: (1) A patch-wise fusion network that fuses a globally-consistent coarse prediction with finer, inconsistent tiled predictions via high-level feature guidance, (2) A Global-to-Local (G2L) module that adds vital context to the fusion network, discarding the need for patch selection heuristics, and (3) A Consistency-Aware Training (CAT) and Inference (CAI) approach, emphasizing patch overlap consistency and thereby eradicating the necessity for post-processing. Experiments on UnrealStereo4K, MVS-Synth, and Middleburry 2014 demonstrate that our framework can generate high-resolution depth maps with intricate details. PatchFusion is independent of the base model for depth estimation. Notably, our framework built on top of SOTA ZoeDepth brings improvements for a total of 17.3% and 29.4% in terms of the root mean squared error (RMSE) on UnrealStereo4K and MVS-Synth, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
