arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2502.09623cs.CV
收藏

在多样NeRF架构上进行权重空间表示学习

Weight Space Representation Learning on Diverse NeRF Architectures

University of Bologna(博洛尼亚大学)

Francesco Ballerini, Pierluigi Zama Ramirez, Luigi Di Stefano, Samuele Salti

AI总结:

本文提出首个能处理多种NeRF架构并进行推理的框架,通过无监督表示学习框架训练图元网络,实现架构无关的潜在空间,实验显示在多种架构上表现稳健。

摘要:

Neural Radiance Fields (NeRFs) 已经成为一种突破性的范式,通过将形状和外观信息编码到神经网络的权重中来表示3D对象和场景。最近的研究表明,这些权重可以作为输入用于解决深度学习任务的框架;然而,此类框架要求NeRFs遵循特定的预定义架构。在本文中,我们引入了首个能够处理多种NeRF架构并进行推理的框架,即使在训练时未曾见过的架构也能进行推理。我们通过在无监督表示学习框架中训练图元网络来实现这一点,并展示对比目标有助于获得架构无关的潜在空间。在13种属于三种家族(MLPs、三平面和首次出现的哈希表)的NeRF架构上进行实验,我们的方法在分类、检索和涉及多种架构的语言任务中表现出稳健的性能,即使这些架构在训练时未曾出现,同时在单架构限制框架上也达到或超越了现有结果。我们的代码和数据可在 https://cvlab-unibo.github.io/gmnerf 上获得。

英文摘要:

Neural Radiance Fields (NeRFs) have emerged as a groundbreaking paradigm for representing 3D objects and scenes by encoding shape and appearance information into the weights of a neural network. Recent studies have demonstrated that these weights can be used as input for frameworks designed to address deep learning tasks; however, such frameworks require NeRFs to adhere to a specific, predefined architecture. In this paper, we introduce the first framework capable of processing NeRFs with diverse architectures and performing inference on architectures unseen at training time. We achieve this by training a Graph Meta-Network within an unsupervised representation learning framework, and show that a contrastive objective is conducive to obtaining an architecture-agnostic latent space. In experiments conducted across 13 NeRF architectures belonging to three families (MLPs, tri-planes, and, for the first time, hash tables), our approach demonstrates robust performance in classification, retrieval, and language tasks involving multiple architectures, even unseen at training time, while also matching or exceeding the results of existing frameworks limited to single architectures. Our code and data are available at https://cvlab-unibo.github.io/gmnerf.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑