In the Spring semester, we conclude that if we have perfect 2D image segmentation, that could benefit the 3D object detection model by a large margin. Given this finding, we propose to improve 2D image segmentation given 3D object detection result. If that is also true, we could close the loop and let both tasks benefit each other. Here is the illustrative figure:
We propose to address two issues simultaneously: (1) fuse 3D information into 2D segmentation (2) sequential inference by cascading the detection model. We highlight the main process in the following two figures:
For details, please refer to our presentation video and slides.