arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.03334cs.CV
收藏

弥合维度差距:2D视觉模型适应3D分析的分类与综述

Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysis

Independent Researcher(独立研究员)

Akshat Pandya, Bhavuk Jain

AI总结:

本文综述了将2D视觉模型适应3D分析的策略,分类为数据导向、架构导向和混合方法,探讨了计算复杂度、预训练依赖性和几何归纳偏置的权衡。

摘要:

卷积神经网络(CNNs)和视觉Transformer(ViTs)在2D视觉中的显著成功推动了将其扩展到3D分析复杂领域的研究。然而,2D图像的常规密集网格与3D数据(如点云和网格)的不规则稀疏性之间存在根本分歧。本文提供了一次全面的回顾和统一的分类,将适应策略分为三类:(1)数据导向方法,将3D数据投影到2D格式以利用现成的2D模型;(2)架构导向方法,设计内在的3D网络;(3)混合方法,结合两种建模范式,利用大规模2D数据集的丰富视觉先验和3D模型的显式几何推理。通过这一框架,我们分析了这些类别在计算复杂性、对大规模预训练的依赖性和几何归纳偏置保持方面的根本权衡。我们讨论了关键的开放挑战,并概述了有前途的未来研究方向,包括开发3D基础模型、几何数据自监督学习(SSL)的进步以及多模态信号的深入整合。

英文摘要:

The remarkable success of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) in 2D vision has spurred significant research in extending these architectures to the complex domain of 3D analysis. Yet, a core challenge arises from a fundamental dichotomy between the regular, dense grids of 2D images and the irregular, sparse nature of 3D data such as point clouds and meshes. This survey provides a comprehensive review and a unified taxonomy of adaptation strategies that bridge this gap, classifying them into three families: (1) Data-centric methods that project 3D data into 2D formats to leverage off-the-shelf 2D models, (2) Architecture-centric methods that design intrinsic 3D networks, and (3) Hybrid methods, which synergistically combine the two modeling paradigms to benefit from both rich visual priors of large 2D datasets and explicit geometric reasoning of 3D models. Through this framework, we qualitatively analyze the fundamental trade-offs between these families concerning computational complexity, reliance on large-scale pre-training, and the preservation of geometric inductive biases. We discuss key open challenges and outline promising future research directions, including the development of 3D foundation models, advancements in self-supervised learning (SSL) for geometric data, and the deeper integration of multi-modal signals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑