NeRF-IS: Explicit Neural Radiance Fields in Semantic Space

Implicit Neural Radiance Field (NeRF) techniques have been widely applied and shown promising results for scene decomposition learning and rendering. Existing methods typically require encoding spatial and semantic coordinates separately, followed by deep neural networks (MLP) to obtain representations of the entire scene and individual objects respectively. However, these implicit neural field methods mix scene data and differentiable rendering together, which results in issues with expensive computation, low interpretability and limited scalability. In this article, we propose NeRF-IS (Explicit Neural Radiance Fields in Semantic Space), a novel 4D neural radiance field model architecture, that integrates 3D space and semantic space modeling, which can perform both scene-level and object-level modeling. Specifically, we design a hybrid method of explicit spatial modeling and implicit feature representation, which enhances the model’s ability in scene semantic editing and realistic rendering. For efficient training of NeRF-IS, we apply low rank tensor decomposition to compress the model and speed up the training. We also introduce an importance sampling algorithm that uses a volume density prediction network to provide more accurate samples for the whole system with a coarse-to-fine strategy. Extensive experiments demonstrate that our system not only achieves competitive performance for scene-level representation and rendering of static scene, but also enables object-level rendering and editing.