{"id":14,"date":"2022-04-29T07:16:19","date_gmt":"2022-04-29T07:16:19","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/?page_id=14"},"modified":"2022-12-18T22:43:44","modified_gmt":"2022-12-18T22:43:44","slug":"introduction","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","title":{"rendered":"Introduction"},"content":{"rendered":"\n
\"\"
Six fisheye camera setup. Three cameras are visible in the blue circles. Similarly, the other three are below the drone, arranged in a triangle opposite to the green one.<\/figcaption><\/figure><\/div>\n\n\n\n

We plan to develop a real-time system deployed on a drone that is capable of estimating the depth of its environment. The drone would have a six fisheye lens camera setup. The six fisheye images would be stitched together to get a panorama image representing the top, down, bottom, left, and right sides of the environment. The model\/system would then predict a depth for each pixel of this stitched image.<\/p>\n\n\n\n

Holonomic robots such as drones have six degrees of freedom and thus, can move in all directions. Human drivers, too, rely on a wider field of view rather than just looking forward to making decisions. In the case of autonomous vehicles and self-driving cars, a LIDAR helps overcome this. However, we want a LIDAR-free system.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Besides being costly, LIDARs are significantly heavy. While in a self-driving car scenario, this doesn’t matter much, in the case of a drone, this would constitute a majority of its payload. Similarly, with a limited power capacity, a LIDAR would end up consuming most of the power on a drone.<\/p>\n\n\n\n

The point cloud produced by a LIDAR gets sparser as the distance increases. Considering the applications of a drone, the distances of the environment, especially top and bottom, could be significantly large. Thus in such cases, the sparse point cloud doesn’t help much. Besides this, LIDAR performs poorly in dusty and rainy conditions.<\/p>\n\n\n\n

Considering these are to be used with a drone, all the LiDAR related issues are doubled as at least 2 such sensors are needed.<\/p>\n\n\n\n

\"\"
Same scene captured by a standard camera lens and a fisheye lens.<\/figcaption><\/figure>\n\n\n\n

We also plan to use fisheye images as these capture more information due to their wider field of view.<\/p>\n\n\n\n

Previous works in multiview stereo either compromise the accuracy or the real-time performance. That is, they are either heavily reliant on powerful GPUs or use some non-learning layers leading to a drop in accuracy.<\/p>\n","protected":false},"excerpt":{"rendered":"

We plan to develop a real-time system deployed on a drone that is capable of estimating the depth of its environment. The drone would have a six fisheye lens camera setup. The six fisheye images would be stitched together to get a panorama image representing the top, down, bottom, left, and right sides of the … <\/p>\n

Continue reading “Introduction”<\/span><\/a><\/p>\n","protected":false},"author":120,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-14","page","type-page","status-publish","hentry"],"yoast_head":"\nIntroduction - Omnidirectional Multi-view Stereo on Edge Devices<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction - Omnidirectional Multi-view Stereo on Edge Devices\" \/>\n<meta property=\"og:description\" content=\"We plan to develop a real-time system deployed on a drone that is capable of estimating the depth of its environment. The drone would have a six fisheye lens camera setup. The six fisheye images would be stitched together to get a panorama image representing the top, down, bottom, left, and right sides of the … Continue reading "Introduction"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/\" \/>\n<meta property=\"og:site_name\" content=\"Omnidirectional Multi-view Stereo on Edge Devices\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-18T22:43:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/12\/Screenshot-from-2022-12-18-17-39-48.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\",\"name\":\"Introduction - Omnidirectional Multi-view Stereo on Edge Devices\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/wp-content\\\/uploads\\\/sites\\\/61\\\/2022\\\/12\\\/Screenshot-from-2022-12-18-17-39-48.png\",\"datePublished\":\"2022-04-29T07:16:19+00:00\",\"dateModified\":\"2022-12-18T22:43:44+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/wp-content\\\/uploads\\\/sites\\\/61\\\/2022\\\/12\\\/Screenshot-from-2022-12-18-17-39-48.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/wp-content\\\/uploads\\\/sites\\\/61\\\/2022\\\/12\\\/Screenshot-from-2022-12-18-17-39-48.png\",\"width\":534,\"height\":404},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\",\"name\":\"Omnidirectional Multi-view Stereo on Edge Devices\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#organization\",\"name\":\"Omnidirectional Multi-view Stereo on Edge Devices\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/wp-content\\\/uploads\\\/sites\\\/61\\\/2022\\\/04\\\/cropped-icon.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/wp-content\\\/uploads\\\/sites\\\/61\\\/2022\\\/04\\\/cropped-icon.png\",\"width\":250,\"height\":250,\"caption\":\"Omnidirectional Multi-view Stereo on Edge Devices\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team6\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction - Omnidirectional Multi-view Stereo on Edge Devices","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","og_locale":"en_US","og_type":"article","og_title":"Introduction - Omnidirectional Multi-view Stereo on Edge Devices","og_description":"We plan to develop a real-time system deployed on a drone that is capable of estimating the depth of its environment. The drone would have a six fisheye lens camera setup. The six fisheye images would be stitched together to get a panorama image representing the top, down, bottom, left, and right sides of the … Continue reading \"Introduction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","og_site_name":"Omnidirectional Multi-view Stereo on Edge Devices","article_modified_time":"2022-12-18T22:43:44+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/12\/Screenshot-from-2022-12-18-17-39-48.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","name":"Introduction - Omnidirectional Multi-view Stereo on Edge Devices","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/12\/Screenshot-from-2022-12-18-17-39-48.png","datePublished":"2022-04-29T07:16:19+00:00","dateModified":"2022-12-18T22:43:44+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/12\/Screenshot-from-2022-12-18-17-39-48.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/12\/Screenshot-from-2022-12-18-17-39-48.png","width":534,"height":404},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/"},{"@type":"ListItem","position":2,"name":"Introduction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","name":"Omnidirectional Multi-view Stereo on Edge Devices","description":"","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#organization","name":"Omnidirectional Multi-view Stereo on Edge Devices","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/04\/cropped-icon.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-content\/uploads\/sites\/61\/2022\/04\/cropped-icon.png","width":250,"height":250,"caption":"Omnidirectional Multi-view Stereo on Edge Devices"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/pages\/14","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/users\/120"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/comments?post=14"}],"version-history":[{"count":3,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/pages\/14\/revisions"}],"predecessor-version":[{"id":122,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/pages\/14\/revisions\/122"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team6\/wp-json\/wp\/v2\/media?parent=14"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}