VQ-NeRF：基于矢量量化的神经反射率分解与编辑
VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization
AI总结：
本文提出VQ-NeRF，通过双分支网络结合矢量量化将三维场景材质离散化，实现低噪声材质分解与交互式离散材质编辑。
摘要：
我们提出VQ-NeRF，一种双分支神经网络模型，该模型引入矢量量化（Vector Quantization, VQ）来分解和编辑三维场景中的反射率场。传统的神经反射率场仅使用连续表示来建模三维场景，尽管现实中的物体通常由离散材质组成。这种离散化的缺失可能导致材质分解产生噪声，并使材质编辑变得复杂。为了解决这些局限性，我们的模型由一个连续分支和一个离散分支组成。连续分支遵循传统流程来预测分解后的材质，而离散分支则使用VQ机制将连续材质量化为独立的单个材质。通过对材质进行离散化，我们的模型能够减少分解过程中的噪声，并生成离散材质的分割图。通过点击分割结果中的相应区域，可以方便地选择特定材质以进行进一步编辑。此外，我们提出了一种基于dropout的VQ码字排序策略来预测场景中的材质数量，从而减少材质分割过程中的冗余。为了提高易用性，我们还开发了一个交互式界面，以进一步辅助材质编辑。我们在计算机生成场景和真实世界场景上评估了我们的模型，展示了其优越的性能。据我们所知，我们的模型是首个能够在三维场景中实现离散材质编辑的模型。
英文摘要：
We propose VQ-NeRF, a two-branch neural network model that incorporates Vector Quantization (VQ) to decompose and edit reflectance fields in 3D scenes. Conventional neural reflectance fields use only continuous representations to model 3D scenes, despite the fact that objects are typically composed of discrete materials in reality. This lack of discretization can result in noisy material decomposition and complicated material editing. To address these limitations, our model consists of a continuous branch and a discrete branch. The continuous branch follows the conventional pipeline to predict decomposed materials, while the discrete branch uses the VQ mechanism to quantize continuous materials into individual ones. By discretizing the materials, our model can reduce noise in the decomposition process and generate a segmentation map of discrete materials. Specific materials can be easily selected for further editing by clicking on the corresponding area of the segmentation outcomes. Additionally, we propose a dropout-based VQ codeword ranking strategy to predict the number of materials in a scene, which reduces redundancy in the material segmentation process. To improve usability, we also develop an interactive interface to further assist material editing. We evaluate our model on both computer-generated and real-world scenes, demonstrating its superior performance. To the best of our knowledge, our model is the first to enable discrete material editing in 3D scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
