Long term visual SLAM faces a number of challenges because of environmental changes over time that are caused by objects that move (furniture, equipment), changes in appearance (open drapes, repaint wall), and apparent changes (lighting, seasonal). We would like to investigate map representations that support collections of map features that may vary over time, efficiently store and index, support efficient loop closure and re-localization, and allow for runing. We would also like to investigate approaches to feature/keyframe management to determine if new representation replaces or augments a previous one, when to merge a new map section, and when to prune an old representation.
The following are our most relevant experiments:
Semantic Maps for Relocalization
In our first experiment, we evaluated the utility of benefit of using Semantic Maps in addition to the PnP+RANSAC algorithm for relocalization. We first used the openLORIS dataset to extract pairs of frames for relocalization along with their ground truth relative pose. This was used to design a simple experiment of the relocalization performance of the PnP+RANSAC method with and without using semantic maps. We found that if we find matches only within the same semantic class, we do better in some cases, but sometimes we lose out on correct matches due to imperfections in the semantic map. However a hybrid approach performs better than both these standalone approaches.
[details]
Depth Based Pruning
Since we were working with an RGBD sensor along with a sparse map representation we implemented a depthmap-based pruning approach for removing unwanted points. We used the depthmap to remove keypoints which were not present in the 3D location where they were projected to be, while preserving occluded and visible objects.
As a result we found a steady decrease in the number of points with a marginal reduction in the average translation error. It also resulted in an occasional improvement in relocalization performance.
[Details]
Datasets
The dataset we used for the relocalization experiment is the OpenLORIS dataset. This dataset is collected for long term SLAM. This dataset contains multiple sequences of the same location with low dynamic changes between them.
For the depth based pruning experiment, we synthesize our sequences on the FRL apartment scenes of the Facebook Replica dataset using FB Habitat Sim.
Advisor
This project is advised by Dr. Michael Kaess
https://www.cs.cmu.edu/~kaess/
Sponsors
This project is sponsored by Amazon
References
Title Image credits: https://github.com/facebookresearch/Replica-Dataset
Openloris Dataset: https://lifelong-robotic-vision.github.io/dataset/scene.html
Facebook Replica Dataset: https://github.com/facebookresearch/Replica-Dataset
ORB-SLAM2: https://github.com/raulmur/ORB_SLAM2
ORB-SLAM3: https://github.com/UZ-SLAMLab/ORB_SLAM3