arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.05617cs.CV
收藏

G-NeLF:用于新视角合成的内存与数据高效的混合神经光场

G-NeLF: Memory- and Data-Efficient Hybrid Neural Light Field for Novel View Synthesis

HKUST(GZ)(香港科技大学(广州));HKUST(香港科技大学)

Lutao Jiang, Lin Wang

AI总结:

针对 NeLF 需依赖 NeRF 与上万视角预训练且渲染质量较低的问题,本文提出基于网格空间感知特征和轻量光线传播解码器的 G-NeLF,以 0.95 MB 模型及十分之一参数超越现有方法。

摘要:

随着隐式神经表示受到日益广泛的关注,Neural Light Field(NeLF,神经光场)被提出用于直接预测一条光线的颜色。与 Neural Radiance Field(NeRF,神经辐射场)不同,NeLF 并不通过预测空间中每个点的颜色和体密度来构建逐点表示。然而,当前 NeLF 方法面临一个挑战:它们需要先训练一个 NeRF 模型,然后合成超过 10K 个视角来训练 NeLF,以获得更好的性能。此外,NeLF 方法的渲染质量低于 NeRF 方法。本文提出 G-NeLF,这是一种通用的基于网格的 NeLF 方法,利用空间感知特征释放神经网络推理能力的潜力,从而克服 NeLF 训练中的困难。具体而言,我们采用由精心设计的网格导出的空间感知特征序列作为光线表示。基于我们对多分辨率哈希表适应性的实证研究,我们为 NeLF 引入了一种新颖的基于网格的光线表示,能够以极少的参数表示整个空间。为了更好地利用序列特征,我们设计了一个模拟光线传播过程的轻量级光线颜色解码器,从而能够更高效地推理光线颜色。G-NeLF 在训练时无需显著的存储开销,并且模型大小仅为 0.95 MB,即可超越此前最先进的 NeLF。此外,与 Instant-NGP 等基于网格的 NeRF 方法相比,我们仅使用其十分之一的参数便实现了更高的性能。我们的代码将在论文被接收后发布。

英文摘要:

Following the burgeoning interest in implicit neural representation, Neural Light Field (NeLF) has been introduced to predict the color of a ray directly. Unlike Neural Radiance Field (NeRF), NeLF does not create a point-wise representation by predicting color and volume density for each point in space. However, the current NeLF methods face a challenge as they need to train a NeRF model first and then synthesize over 10K views to train NeLF for improved performance. Additionally, the rendering quality of NeLF methods is lower compared to NeRF methods. In this paper, we propose G-NeLF, a versatile grid-based NeLF approach that utilizes spatial-aware features to unleash the potential of the neural network's inference capability, and consequently overcome the difficulties of NeLF training. Specifically, we employ a spatial-aware feature sequence derived from a meticulously crafted grid as the ray's representation. Drawing from our empirical studies on the adaptability of multi-resolution hash tables, we introduce a novel grid-based ray representation for NeLF that can represent the entire space with a very limited number of parameters. To better utilize the sequence feature, we design a lightweight ray color decoder that simulates the ray propagation process, enabling a more efficient inference of the ray's color. G-NeLF can be trained without necessitating significant storage overhead and with the model size of only 0.95 MB to surpass previous state-of-the-art NeLF. Moreover, compared with grid-based NeRF methods, e.g., Instant-NGP, we only utilize one-tenth of its parameters to achieve higher performance. Our code will be released upon acceptance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑