Conditional Neural Fields (CNFs) are increasingly being leveraged as continuous signal representations, by associating each data-sample with a latent variable that conditions a shared backbone Neural Field (NeF) to reconstruct the sample. However, existing CNF architectures face limitations when using this latent downstream in tasks requiring fine-grained geometric reasoning, such as classification and segmentation. We posit that this results from lack of explicit modelling of geometric information (e.g., locality in the signal or the orientation of a feature) in the latent space of CNFs. As such, we propose Equivariant Neural Fields (ENFs), a novel CNF architecture which uses a geometry-informed cross-attention to condition the NeF on a geometric variable--a latent point cloud of features--that enables an equivariant decoding from latent to field. We show that this approach induces a steerability property by which both field and latent are grounded in geometry and amenable to transformation laws: if the field transforms, the latent representation transforms accordingly--and vice versa. Crucially, this equivariance relation ensures that the latent is capable of (1) representing geometric patterns faithfully, allowing for geometric reasoning in latent space, and (2) weight-sharing over similar local patterns, allowing for efficient learning of datasets of fields. We validate these main properties in a range of tasks including classification, segmentation, forecasting, reconstruction and generative modelling, showing clear improvement over baselines with a geometry-free latent space. Code attached to submission https://github.com/Dafidofff/enf-jax. Code for a clean and minimal repo https://github.com/david-knigge/enf-min-jax.
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);Fudan University(复旦大学)
Amsterdam Machine Learning Lab (AMLab), University of Amsterdam(阿姆斯特丹机器学习实验室(AMLab),阿姆斯特丹大学);New Theory(新理论);Department of Mathematics and Computer Science, Eindhoven University of Technology(数学与计算机科学系,埃因霍温理工大学);Leiden Institute of Advanced Computer Science, Universiteit Leiden(莱顿高级计算机科学研究所,莱顿大学)
PERM EQ x GRAPH EQ: Equivariant Neural Networks for Quantum Molecular Learning
Independent Researcher, College Park, Maryland, United States(独立研究者,马里兰州学院公园,美国);Independent AI Researcher, San Francisco, California, United States(独立AI研究者,加利福尼亚州旧金山,美国)