arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2607.02707cs.CV

VLRC:视觉语言重投影一致性作为用于更好的前馈3D预训练的可扩展信号

VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining

U2IS, ENSTA – Institut Polytechnique de Paris(U2IS,法国国立高等先进技术学校 - 巴黎综合理工学院);Pôle Recherche, Agence Ministérielle pour l’IA de Défense(军事人工智能部研究中心)

Marwane Hariat, David Filliat, Antoine Manzanera

AI总结:

研究提出视觉语言重投影一致性(VLRC),利用冻结的视觉语言表征作为语义多视图监督,无需额外3D标注,能提升3D重建精度等,为前馈3D预训练提供可扩展辅助目标。

摘要:

前馈3D模型常用昂贵几何监督或自监督光度目标训练,信号不完整。我们引入视觉语言重投影一致性(VLRC),利用冻结视觉语言表征作为语义多视图监督。给定预测3D重建,VLRC跨视图重投影密集视觉语言特征并强制对应图像位置特征一致,无需额外3D标注。该目标在无标签适应期间与自监督单目重建和监督预训练前馈3D模型无缝集成。通过将几何与基于语言的特征对齐,VLRC不仅提高深度和相机估计,还能实现更连贯的多视图语义融合,用于开放词汇3D场景理解。室内和室外基准实验证明在3D重建精度和零样本开放词汇3D语义分割方面持续提升。

英文摘要:

Feed-forward 3D models are commonly trained using either expensive geometric supervision or self-supervised photometric objectives, both of which provide incomplete learning signals. We introduce Vision-Language Reprojection Consistency (VLRC), a scalable auxiliary objective that exploits frozen vision-language representations as semantic multi-view supervision. Given a predicted 3D reconstruction, VLRC reprojects dense vision-language features across views and enforces feature consistency between corresponding image locations, requiring no additional 3D annotations. The objective integrates seamlessly with both self-supervised monocular reconstruction and supervised-pretrained feed-forward 3D models during unlabeled adaptation. By aligning geometry with language-grounded features, VLRC not only improves depth and camera estimation but also enables more coherent multi-view semantic fusion for open-vocabulary 3D scene understanding. Experiments on indoor and outdoor benchmarks demonstrate consistent gains in 3D reconstruction accuracy and zero-shot open-vocabulary 3D semantic segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑