arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.05753cs.LGcs.AIcs.CV
收藏

在几何中锚定连续表示:等变神经场

Grounding Continuous Representations in Geometry: Equivariant Neural Fields

University of Amsterdam(阿姆斯特丹大学)

David R Wessels, David M Knigge, Samuele Papa, Riccardo Valperga, Sharvaree Vadgama, Efstratios Gavves, Erik J Bekkers

AI总结:

本文提出等变神经场(ENFs),通过几何信息引导的交叉注意力将条件神经场锚定在潜在特征点云上,实现潜在与场之间的等变解码,从而增强潜在空间的几何推理能力和权重共享效率,并在分类、分割、预测、重建和生成建模等任务中验证其优势。

摘要:

条件神经场(CNFs)正越来越多地被用作连续信号表示,其方式是将每个数据样本与一个潜在变量相关联,该潜在变量条件化一个共享的主干神经场(NeF)以重建样本。然而,现有的CNF架构在将这种潜在变量用于需要细粒度几何推理的下游任务(如分类和分割)时面临局限。我们认为,这是因为CNF的潜在空间缺乏对几何信息(例如信号中的局部性或特征的方向)的显式建模。因此,我们提出了等变神经场(ENFs),这是一种新颖的CNF架构,它使用几何信息引导的交叉注意力,将NeF条件化在一个几何变量上——一个潜在的特征点云——从而实现从潜在到场的等变解码。我们表明,这种方法诱导出一种可操控性,使得场和潜在表示都锚定在几何中并服从变换法则:如果场发生变换,潜在表示也相应变换——反之亦然。至关重要的是,这种等变关系确保了潜在表示能够(1)忠实地表示几何模式,从而允许在潜在空间中进行几何推理,以及(2)在相似的局部模式上实现权重共享,从而允许对场数据集进行高效学习。我们在包括分类、分割、预测、重建和生成建模在内的一系列任务中验证了这些主要性质,与具有无几何潜在空间的基线相比,显示出明显的改进。代码附于提交 https://github.com/Dafidofff/enf-jax。干净且最小化仓库的代码 https://github.com/david-knigge/enf-min-jax。

英文摘要:

Conditional Neural Fields (CNFs) are increasingly being leveraged as continuous signal representations, by associating each data-sample with a latent variable that conditions a shared backbone Neural Field (NeF) to reconstruct the sample. However, existing CNF architectures face limitations when using this latent downstream in tasks requiring fine-grained geometric reasoning, such as classification and segmentation. We posit that this results from lack of explicit modelling of geometric information (e.g., locality in the signal or the orientation of a feature) in the latent space of CNFs. As such, we propose Equivariant Neural Fields (ENFs), a novel CNF architecture which uses a geometry-informed cross-attention to condition the NeF on a geometric variable--a latent point cloud of features--that enables an equivariant decoding from latent to field. We show that this approach induces a steerability property by which both field and latent are grounded in geometry and amenable to transformation laws: if the field transforms, the latent representation transforms accordingly--and vice versa. Crucially, this equivariance relation ensures that the latent is capable of (1) representing geometric patterns faithfully, allowing for geometric reasoning in latent space, and (2) weight-sharing over similar local patterns, allowing for efficient learning of datasets of fields. We validate these main properties in a range of tasks including classification, segmentation, forecasting, reconstruction and generative modelling, showing clear improvement over baselines with a geometry-free latent space. Code attached to submission https://github.com/Dafidofff/enf-jax. Code for a clean and minimal repo https://github.com/david-knigge/enf-min-jax.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑