{"id":7,"date":"2023-05-08T20:55:28","date_gmt":"2023-05-08T20:55:28","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/?page_id=7"},"modified":"2023-12-18T22:02:06","modified_gmt":"2023-12-18T22:02:06","slug":"setup","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/","title":{"rendered":"Setup &amp; Dataset"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\">Experiment Setup<\/h1>\n\n\n\n<p>Our experiment setup involves a drone equipped with three fisheye cameras mounted on the top and additional three fisheye cameras mounted on the bottom. <\/p>\n\n\n\n<p class=\"has-text-align-center\"><br><img loading=\"lazy\" decoding=\"async\" width=\"589\" height=\"468\" class=\"wp-image-105\" style=\"width: 200px\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg\" alt=\"\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg 589w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3-300x238.jpg 300w\" sizes=\"auto, (max-width: 589px) 100vw, 589px\" \/><\/p>\n\n\n\n<p>The fisheye lenses used in the experiment are known to have barrel (radial) distortion. They offer a <strong>wider<\/strong> field of view (<strong>FoV<\/strong>), providing access to <strong>more information<\/strong>. However, due to the distortion, useful information may be <strong>cramped on the edges<\/strong> of the captured images.<\/p>\n\n\n\n<p class=\"has-text-align-center\"><img loading=\"lazy\" decoding=\"async\" width=\"787\" height=\"230\" class=\"wp-image-106\" style=\"width: 450px\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-4.jpg\" alt=\"\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-4.jpg 787w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-4-300x88.jpg 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-4-768x224.jpg 768w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/p>\n\n\n\n<p>There are a few ways to deal with the high levels of distortion in fisheye space &#8211; one way is to transform the input fisheye into an equirectangular or a cube-map space. The most common equirectangular projection is done by unwrapping the fisheye along an axis (in red in the below image).<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"814\" height=\"322\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/12\/image.png\" alt=\"\" class=\"wp-image-195\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/12\/image.png 814w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/12\/image-300x119.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/12\/image-768x304.png 768w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p> It can observed that the majority of the center of the fisheye maps to a small region on the unwrapped equirecntagular image (in the yellow line). The cramped edge regions from the input fisheye are mapped to the majority of the remainder of the image (in the dotted blue line).<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Data Collection<\/h1>\n\n\n\n<p><strong>Dataset analysis<\/strong>: TartanAir dataset collected in\u00a0<strong>Airsim<\/strong>\u00a0simulator (running on Unreal Engine).<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>28 environments: ~45 videos; ~4500 valid frames<\/li>\n\n\n\n<li>Per frame samples: 3&nbsp;fisheye camera RGBs, Distance GTs, and Rig RGBs<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"756\" height=\"711\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-5.jpg\" alt=\"\" class=\"wp-image-107\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-5.jpg 756w, https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-5-300x282.jpg 300w\" sizes=\"auto, (max-width: 706px) 89vw, (max-width: 767px) 82vw, 740px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Experiment Setup Our experiment setup involves a drone equipped with three fisheye cameras mounted on the top and additional three fisheye cameras mounted on the bottom. The fisheye lenses used in the experiment are known to have barrel (radial) distortion. They offer a wider field of view (FoV), providing access to more information. However, due &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Setup &amp; Dataset&#8221;<\/span><\/a><\/p>\n","protected":false},"author":182,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-7","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Setup &amp; Dataset - Multi-view robot perception on Edge Devices<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Setup &amp; Dataset - Multi-view robot perception on Edge Devices\" \/>\n<meta property=\"og:description\" content=\"Experiment Setup Our experiment setup involves a drone equipped with three fisheye cameras mounted on the top and additional three fisheye cameras mounted on the bottom. The fisheye lenses used in the experiment are known to have barrel (radial) distortion. They offer a wider field of view (FoV), providing access to more information. However, due &hellip; Continue reading &quot;Setup &amp; Dataset&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/\" \/>\n<meta property=\"og:site_name\" content=\"Multi-view robot perception on Edge Devices\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-18T22:02:06+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/\",\"name\":\"Setup &amp; Dataset - Multi-view robot perception on Edge Devices\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/16824_project_graphs.pptx-3.jpg\",\"datePublished\":\"2023-05-08T20:55:28+00:00\",\"dateModified\":\"2023-12-18T22:02:06+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/#primaryimage\",\"url\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/16824_project_graphs.pptx-3.jpg\",\"contentUrl\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/16824_project_graphs.pptx-3.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/setup\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Setup &amp; Dataset\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"name\":\"Multi-view robot perception on Edge Devices\",\"description\":\"CMU MSCV &#039;23 capstone project\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#organization\",\"name\":\"Multi-view robot perception on Edge Devices\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/cropped-cropped-icon.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/cropped-cropped-icon.png\",\"width\":250,\"height\":250,\"caption\":\"Multi-view robot perception on Edge Devices\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Setup &amp; Dataset - Multi-view robot perception on Edge Devices","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/","og_locale":"en_US","og_type":"article","og_title":"Setup &amp; Dataset - Multi-view robot perception on Edge Devices","og_description":"Experiment Setup Our experiment setup involves a drone equipped with three fisheye cameras mounted on the top and additional three fisheye cameras mounted on the bottom. The fisheye lenses used in the experiment are known to have barrel (radial) distortion. They offer a wider field of view (FoV), providing access to more information. However, due &hellip; Continue reading \"Setup &amp; Dataset\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/","og_site_name":"Multi-view robot perception on Edge Devices","article_modified_time":"2023-12-18T22:02:06+00:00","og_image":[{"url":"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/","name":"Setup &amp; Dataset - Multi-view robot perception on Edge Devices","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/#primaryimage"},"thumbnailUrl":"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg","datePublished":"2023-05-08T20:55:28+00:00","dateModified":"2023-12-18T22:02:06+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/#primaryimage","url":"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg","contentUrl":"http:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/16824_project_graphs.pptx-3.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/setup\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/"},{"@type":"ListItem","position":2,"name":"Setup &amp; Dataset"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","name":"Multi-view robot perception on Edge Devices","description":"CMU MSCV &#039;23 capstone project","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#organization","name":"Multi-view robot perception on Edge Devices","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/cropped-cropped-icon.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/cropped-cropped-icon.png","width":250,"height":250,"caption":"Multi-view robot perception on Edge Devices"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/7","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/users\/182"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":11,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/7\/revisions"}],"predecessor-version":[{"id":206,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/7\/revisions\/206"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/media?parent=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}