重新审视极化形变在视觉基础模型时代的应用
Revisiting Shape from Polarization in the Era of Vision Foundation Models
AI总结：
本研究通过极化线索和改进的数据增强方法，在少量训练数据下实现了优于传统RGB模型的表面法线估计性能。
摘要：
我们证明，借助极化线索，一个在小数据集上训练的轻量模型可以在单次物体级表面法线估计中超越仅使用RGB的视觉基础模型（VFMs）。极化形变（SfP）长期以来一直受到极化与表面几何之间强物理关系的推动。同时，受缩放定律驱动，仅使用RGB的VFMs在大数据库上训练后最近取得了令人印象深刻的性能，并超越了现有的SfP方法。这种状况引发了关于极化线索必要性的疑问，因为极化线索需要专用硬件且训练数据有限。我们认为，先前SfP方法的较弱性能并非来自极化模态本身，而是来自领域差距。这些领域差距主要来自两个来源。首先，现有合成数据集使用有限且不现实的3D物体，具有简单的几何结构和随机纹理图，这些不匹配基本形状。其次，现实世界中的极化信号常受传感器噪声影响，这在训练过程中没有得到良好建模。为了解决第一个问题，我们使用1954个3D扫描的现实世界物体渲染高质量的极化数据集。我们进一步结合预训练的DINOv3先验知识以提高对未见过物体的泛化能力。为了解决第二个问题，我们引入了极化传感器感知的数据增强，以更好地反映现实条件。仅用40K训练场景，我们的方法显著优于最先进的SfP方法和仅使用RGB的VFMs。广泛的实验表明，极化线索使训练数据减少33倍或模型参数减少8倍，同时仍能实现比仅使用RGB的对应方法更好的性能。
英文摘要：
We show that, with polarization cues, a lightweight model trained on a small dataset can outperform RGB-only vision foundation models (VFMs) in single-shot object-level surface normal estimation. Shape from polarization (SfP) has long been studied due to the strong physical relationship between polarization and surface geometry. Meanwhile, driven by scaling laws, RGB-only VFMs trained on large datasets have recently achieved impressive performance and surpassed existing SfP methods. This situation raises questions about the necessity of polarization cues, which require specialized hardware and have limited training data. We argue that the weaker performance of prior SfP methods does not come from the polarization modality itself, but from domain gaps. These domain gaps mainly arise from two sources. First, existing synthetic datasets use limited and unrealistic 3D objects, with simple geometry and random texture maps that do not match the underlying shapes. Second, real-world polarization signals are often affected by sensor noise, which is not well modeled during training. To address the first issue, we render a high-quality polarization dataset using 1,954 3D-scanned real-world objects. We further incorporate pretrained DINOv3 priors to improve generalization to unseen objects. To address the second issue, we introduce polarization sensor-aware data augmentation that better reflects real-world conditions. With only 40K training scenes, our method significantly outperforms both state-of-the-art SfP approaches and RGB-only VFMs. Extensive experiments show that polarization cues enable a 33x reduction in training data or an 8x reduction in model parameters, while still achieving better performance than RGB-only counterparts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
