Neural radiance fields (NeRF) have demonstrated the potential of coordinate-based neural representation (neural fields or implicit neural representation) in neural rendering. However, using a multi-layer perceptron (MLP) to represent a 3D scene or object requires enormous computational resources and time. There have been recent studies on how to reduce these computational inefficiencies by using additional data structures, such as grids or trees. Despite the promising performance, the explicit data structure necessitates a substantial amount of memory. In this work, we present a method to reduce the size without compromising the advantages of having additional data structures. In detail, we propose using the wavelet transform on grid-based neural fields. Grid-based neural fields are for fast convergence, and the wavelet transform, whose efficiency has been demonstrated in high-performance standard codecs, is to improve the parameter efficiency of grids. Furthermore, in order to achieve a higher sparsity of grid coefficients while maintaining reconstruction quality, we present a novel trainable masking approach. Experimental results demonstrate that non-spatial grid coefficients, such as wavelet coefficients, are capable of attaining a higher level of sparsity than spatial grid coefficients, resulting in a more compact representation. With our proposed mask and compression pipeline, we achieved state-of-the-art performance within a memory budget of 2 MB. Our code is available at https://github.com/daniel03c1/masked_wavelet_nerf.
Nanyang Technological University（南洋理工大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Wenzhou University（温州大学）；UCAS-Terminus AI Lab, UCAS（中国科学院大学-特米纳斯人工智能实验室，中国科学院大学）
NeRF-MIR: Towards High-Quality Restoration of Masked Images with Neural Radiance Fields
Shanghai Key Lab of Intelligent Information Processing（上海智能信息处理重点实验室）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Jiao Tong University（上海交通大学）
Learning from the Right Patches: A Two-Stage Wavelet-Driven Masked Autoencoder for Histopathology Representation Learning
PLRI Medical Informatics Institute（PLRI医学信息学研究所）；CAIMed Reserch Center（CAIMed研究中心）；Hannover Medical School（汉诺威医学院）；Computer Science Institute（计算机科学研究所）；Leibniz University Hannover（莱比锡大学汉诺威分校）；Sanford Burnham Prebys Medical Discovery Institute（桑福医疗发现研究所）；Rady Children’s Institute for Genomic Medicine（拉迪儿童基因医学研究所）；University of California San Diego（加州大学圣地亚哥分校）
SpikingNeRF: Making Bio-inspired Neural Networks See through the Real World
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；China Electric Power Research Institute Co., Ltd（中国电力科学研究院有限公司）
School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China（南昌大学数学与计算机科学学院）；School of Information Engineering, Nanchang University, Nanchang, China（南昌大学信息工程学院）；Key Laboratory of Advanced Medical Imaging and Intelligent Computing of Guizhou Province, China（贵州省先进医学影像与智能计算重点实验室）
Wavelet-Domain Masked Image Modeling for Color-Consistent HDR Video Reconstruction
School of Computer Science and Technology and the Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University（计算机科学与技术学院和嵌入式系统与服务计算重点实验室，教育部，同济大学）；College of Electronic and Information Engineering, the School of Computer Science and Technology, and the Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University（电子与信息工程学院、计算机科学与技术学院和嵌入式系统与服务计算重点实验室，教育部，同济大学）；Pengcheng Laboratory（鹏城实验室）
Implicit neural representations (INRs) have recently advanced numerous vision-related areas. INR performance depends strongly on the choice of the nonlinear activation function employed in its multilayer perceptron (MLP) network. A wide range of nonlinearities have been explored, but, unfortunately, current INRs designed to have high accuracy also suffer from poor robustness (to signal noise, parameter variation, etc.). Inspired by harmonic analysis, we develop a new, highly accurate and robust INR that does not exhibit this tradeoff. Wavelet Implicit neural REpresentation (WIRE) uses a contin…
Technical University of Munich（慕尼黑工业大学）；Meta Reality Labs Zurich（Meta苏黎世现实实验室）
We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models. While existing diffusion-based methods operate on images, latent codes, or point cloud data, we are the first to directly generate volumetric radiance fields. To this end, we propose a 3D denoising model which directly operates on an explicit voxel grid representation. However, as radiance fields generated from a set of posed images can be ambiguous and contain artifacts, obtaining ground truth radiance field samples is non-trivial. We address this challenge by pairing the d…
FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；Vivo（vivo公司）；School of Mathematics, Nanjing University（南京大学数学学院）；Department of Computer Science and Technology, Nanjing University（南京大学计算机科学与技术系）
