Clear- Splatting builds on prior work in novel view synthesis to render depth maps from the 3D reconstruction. A popular novel-view synthesis approach is NeRF [7], which uses neural networks to learn a mapping from a 3D point and view angle to a density and an RGB radiance. NeRF renders pixels using existing volume rendering techniques. Subsequent works improve NeRF along several axes: e.g., speeding up training and inference time via novel representations and system optimizations [11], [12], [13], [14], [15], [16], [17], [11], [18], or depth supervision [19], [20], [21], [22], [23]. Other works extend NeRF to more challenging conditions, such as sparser camera views [24], [25], [26], [27], fewer extrinsic camera calibrations [28], [29], [30], [31], transparent objects [8], [1], [9] and reflective surfaces [32].

Several works have proposed methods for accurate depth perception, shape estimation, and/or pose estimation. Xie et al. [33] developed a pipeline based on transformer neural networks capable of transparent object segmentation. Phillips et al. [3] leveraged a random forest algorithm to extract the pose and shape of transparent objects. Xu et al. [4] contributed an algorithm for estimating the 6-degrees-of-freedom (DOF) pose of a transparent object using only a single RGBD image. Wang et al. [34] contributed MVTrans for depth mapping, segmentation, and pose estimation of transparent objects. Chen et al. [2] contributed a benchmark dataset for segmentation, object pose estimation, and depth completion.

Dex-NeRF

Ichnowski et al. [1] showed how NeRFs can be leveraged to infer state-of-the-art depth perception of transparent objects, and unlike training depth supervision-centric approaches, did not require prior training on a set of objects.

Residual-NeRF

Residual-NeRF [9] is a NeRF-based method to capture depth from scenes containing transparent objects. They contributed a method which uses a background NeRF, a Residual-NeRF, and a Mix-Net to speed up training and improve depth maps. The background NeRF is trained just on the background images and this is taken as an initialization when training the Residual-NeRF on the images containing the object with the scene.

3D Gaussian Splatting

3D Gaussian Splatting [10] proposed a differential rasterizer to render a large number of Gaussian Splats, each with their state including color, position, and covariance matrix. Clear-Splatting builds on 3D Gaussian Splatting for better depth rendering.