Problem Statement
Given a set of cameras placed at a distance create a composite panoramic image by blending/ stitching the images captured from the cameras.
Solution
There are currently 2 solution approaches considered for the task of image stitching.
Homography Matrix estimation.
This is a simple and effective way to generate panoramic images.
Here are the steps to create a Panoramic image using the Homography matrix:
– To find feature correspondences/keypoints between the images and match those keypoints present in the overlapping area of two images. I have decided to use the Superpoint keypoint detector and descriptor and SuperGlue feature matching model for this project.
– Obtain the homography matrix from the two points using SVD
– Warp one image onto another using the Homography matrix
– Use image blending techniques to remove seams and other artifacts and make a panoramic image more visually realistic
Though this technique is fairly simple and produces good results, one of the key assumptions of this technique is that there should be only rotational motion between the cameras used to obtain the images or the image should be of a planar surface. As we have cameras placed at different locations, there is translation motion between the cameras along with some possible rotation. The surface of image capture is almost planar this method will produce average results.
Results
As-Projective-As-Possible Image Stitching with Moving DLT
The solution proposed in the paper[1] address to solve the problem of stitching images when assumptions of homography are broken. Since our images are neither planar and there is translation present between images, we break the assumptions for Homography and as a result obtain poorly stitched image.
In this paper they propose a concept of of having local homography instead of one single global homography and warp the images based on the local homography. Local homopgraphy is obtained by weighting the DLT while computing DLT based of its location from keypoints. The paper uses SIFT for keypoint detection and matching. I have used SuperPoint and SuperGlue/ LoFTR algorithms to find better and more diverse keypoints.
Results
Note:
While this method aligns the planar region of the image(wall) very well, it fails to work on non-planar parts of the image and weird overlapping artifacts are created, There is some overlapping misalignment at the end of image 2(image on right) overlap over image 1 (on left), this can be attributed to the poor un-distortion parameters.
Reference
[1] Zaragoza, Julio, et al. “As-projective-as-possible image stitching with moving DLT.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2013.