{"id":19,"date":"2022-05-01T04:39:08","date_gmt":"2022-05-01T04:39:08","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/?page_id=19"},"modified":"2022-12-19T20:07:12","modified_gmt":"2022-12-19T20:07:12","slug":"image-stitching","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/","title":{"rendered":"Image Stitching"},"content":{"rendered":"\n

Problem Statement<\/strong><\/h2>\n\n\n\n

Given a set of cameras placed at a distance create a composite panoramic image by blending\/ stitching the images captured from the cameras.<\/p>\n\n\n\n

\"How
credits: https:\/\/www.panoramic-photo-guide.com<\/figcaption><\/figure>\n\n\n\n

Solution<\/strong><\/h3>\n\n\n\n

There are currently 2 solution approaches considered for the task of image stitching.<\/p>\n\n\n\n

Homography Matrix estimation.<\/strong><\/h3>\n\n\n\n

This is a simple and effective way to generate panoramic images.<\/p>\n\n\n\n

Here are the steps to create a Panoramic image using the Homography matrix:<\/p>\n\n\n\n

– To find feature correspondences\/keypoints between the images and match those keypoints present in the overlapping area of two images. I have decided to use the Superpoint keypoint detector and descriptor and SuperGlue feature matching model for this project.<\/p>\n\n\n\n

– Obtain the homography matrix from the two points using SVD<\/p>\n\n\n\n

– Warp one image onto another using the Homography matrix<\/p>\n\n\n\n

– Use image blending techniques to remove seams and other artifacts and make a panoramic image more visually realistic<\/p>\n\n\n\n

Though this technique is fairly simple and produces good results, one of the key assumptions of this technique is that there should be only rotational motion between the cameras used to obtain the images or the image should be of a planar surface. As we have cameras placed at different locations, there is translation motion between the cameras along with some possible rotation. The surface of image capture is almost planar this method will produce average results.<\/p>\n\n\n\n

Results<\/h4>\n\n\n\n
\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n
Input image pair<\/figcaption><\/figure>\n\n\n\n
\"\"
Poor Result from normal Homography <\/figcaption><\/figure>\n\n\n\n

As-Projective-As-Possible Image Stitching with Moving DLT<\/strong><\/h3>\n\n\n\n

The solution proposed in the paper[1] address to solve the problem of stitching images when assumptions of homography are broken. Since our images are neither planar and there is translation present between images, we break the assumptions for Homography and as a result obtain poorly stitched image.
In this paper they propose a concept of of having local homography instead of one single global homography and warp the images based on the local homography. Local homopgraphy is obtained by weighting the DLT while computing DLT based of its location from keypoints. The paper uses SIFT for keypoint detection and matching. I have used SuperPoint and SuperGlue\/ LoFTR algorithms to find better and more diverse keypoints.<\/p>\n\n\n\n

Results<\/h4>\n\n\n\n
\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n
Input image pair<\/figcaption><\/figure>\n\n\n\n
\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n
Keypoints using LoFTR<\/figcaption><\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Note:
While this method aligns the planar region of the image(wall) very well, it fails to work on non-planar parts of the image and weird overlapping artifacts are created, There is some overlapping misalignment at the end of image 2(image on right) overlap over image 1 (on left), this can be attributed to the poor un-distortion parameters.<\/p>\n\n\n\n

Reference<\/h4>\n\n\n\n

[1] Zaragoza, Julio, et al. “As-projective-as-possible image stitching with moving DLT.”\u00a0Proceedings of the IEEE conference on computer vision and pattern recognition<\/em>. 2013.<\/p>\n","protected":false},"excerpt":{"rendered":"

Problem Statement Given a set of cameras placed at a distance create a composite panoramic image by blending\/ stitching the images captured from the cameras. Solution There are currently 2 solution approaches considered for the task of image stitching. Homography Matrix estimation. This is a simple and effective way to generate panoramic images. Here are … <\/p>\n

Continue reading “Image Stitching”<\/span><\/a><\/p>\n","protected":false},"author":123,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-19","page","type-page","status-publish","hentry"],"yoast_head":"\nImage Stitching - Multi-Camera Pose Estimation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Image Stitching - Multi-Camera Pose Estimation\" \/>\n<meta property=\"og:description\" content=\"Problem Statement Given a set of cameras placed at a distance create a composite panoramic image by blending\/ stitching the images captured from the cameras. Solution There are currently 2 solution approaches considered for the task of image stitching. Homography Matrix estimation. This is a simple and effective way to generate panoramic images. Here are … Continue reading "Image Stitching"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/\" \/>\n<meta property=\"og:site_name\" content=\"Multi-Camera Pose Estimation\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-19T20:07:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.panoramic-photo-guide.com\/images\/stitch-a-panorama\/principle-stitching.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/\",\"name\":\"Image Stitching - Multi-Camera Pose Estimation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.panoramic-photo-guide.com\\\/images\\\/stitch-a-panorama\\\/principle-stitching.jpg\",\"datePublished\":\"2022-05-01T04:39:08+00:00\",\"dateModified\":\"2022-12-19T20:07:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.panoramic-photo-guide.com\\\/images\\\/stitch-a-panorama\\\/principle-stitching.jpg\",\"contentUrl\":\"https:\\\/\\\/www.panoramic-photo-guide.com\\\/images\\\/stitch-a-panorama\\\/principle-stitching.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/image-stitching\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Image Stitching\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/\",\"name\":\"Multi-Camera Pose Estimation\",\"description\":\"Student: Aditya Ghuge | Advisor: Fernando De La Torre | Sponsor: Komatsu\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team12\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Image Stitching - Multi-Camera Pose Estimation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/","og_locale":"en_US","og_type":"article","og_title":"Image Stitching - Multi-Camera Pose Estimation","og_description":"Problem Statement Given a set of cameras placed at a distance create a composite panoramic image by blending\/ stitching the images captured from the cameras. Solution There are currently 2 solution approaches considered for the task of image stitching. Homography Matrix estimation. This is a simple and effective way to generate panoramic images. Here are … Continue reading \"Image Stitching\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/","og_site_name":"Multi-Camera Pose Estimation","article_modified_time":"2022-12-19T20:07:12+00:00","og_image":[{"url":"https:\/\/www.panoramic-photo-guide.com\/images\/stitch-a-panorama\/principle-stitching.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/","name":"Image Stitching - Multi-Camera Pose Estimation","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/#primaryimage"},"thumbnailUrl":"https:\/\/www.panoramic-photo-guide.com\/images\/stitch-a-panorama\/principle-stitching.jpg","datePublished":"2022-05-01T04:39:08+00:00","dateModified":"2022-12-19T20:07:12+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/#primaryimage","url":"https:\/\/www.panoramic-photo-guide.com\/images\/stitch-a-panorama\/principle-stitching.jpg","contentUrl":"https:\/\/www.panoramic-photo-guide.com\/images\/stitch-a-panorama\/principle-stitching.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/image-stitching\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/"},{"@type":"ListItem","position":2,"name":"Image Stitching"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/","name":"Multi-Camera Pose Estimation","description":"Student: Aditya Ghuge | Advisor: Fernando De La Torre | Sponsor: Komatsu","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/pages\/19","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/users\/123"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/comments?post=19"}],"version-history":[{"count":9,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/pages\/19\/revisions"}],"predecessor-version":[{"id":175,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/pages\/19\/revisions\/175"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team12\/wp-json\/wp\/v2\/media?parent=19"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}