Novel view reconstructions for (left) our method and (right) conventional 3D Gaussian Splatting with random initializations. Our method, even with random initialization, faithfully reconstructs the scene (e.g.. buildings at the back and the ground texture) providing much higher quality renderings.
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which does not always generalize and may lead to poor-quality renderings. For many real-world scenes this leads to their heavy dependence on good initializations. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physical representation of the scene---in other words, Markov Chain Monte Carlo (MCMC) samples. Under this view, we show that the 3D Gaussian updates can be converted as Stochastic Gradient Langevin Dynamics (SGLD) update by simply introducing noise. We then rewrite the densification and pruning strategies in 3D Gaussian Splatting as simply a deterministic state transition of MCMC samples, removing these heuristics from the framework. To do so, we revise the `cloning' of Gaussians into a relocalization scheme that approximately preserves distribution invariance. To encourage using fewer Gaussians for efficiency, we introduce an L1-regularizer on the Gaussians. On various standard evaluation scenes, we show that our method provides improved rendering quality, easy control over the number of Gaussians, and robustness to initialization.
| '10' sequence from OMMO dataset | |||
| 3DGS-Random | 3DGS | ||
| Ours-Random | Ours | ||
| 'Stump' sequence from the MipNeRF360 dataset (pay attention to the details between the leaves) | |||
| 3DGS-Random | 3DGS | ||
| Ours-Random | Ours | ||