{"id":2,"date":"2023-05-01T17:48:32","date_gmt":"2023-05-01T17:48:32","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/?page_id=2"},"modified":"2023-12-18T22:03:47","modified_gmt":"2023-12-18T22:03:47","slug":"sample-page","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","title":{"rendered":"Introduction"},"content":{"rendered":"\n

Depth Estimation<\/p>\n\n\n\n

Holonomic robots such as drones have six degrees of freedom and thus, can move in all directions. Human drivers, too, rely on a wider field of view rather than just looking forward to making decisions. In the case of autonomous vehicles<\/strong> and self-driving cars<\/strong>, a LIDAR helps overcome this. <\/p>\n\n\n\n

However, LiDAR faces challenges in various scenarios<\/strong> such as the SEARCH scenario where accurate depth estimation is required for obstacle detection, and in ADAS scenario, it struggles with detecting dark or shiny objects, and the dazzling phenomenon. For mapping and obstacle avoidance in holonomic robots like drones, LiDAR’s limitations include high form-factor, heavy weight, high power consumption, limited field of view, and sparse LiDAR points per object. Additionally, cost can be a significant drawback with multiple LiDARs required per vehicle.<\/p>\n\n\n\n

Therefore, we want a LIDAR-free system<\/strong>. We plan to use fisheye images<\/strong> as these capture more information due to their wider field of view<\/strong>.<\/p>\n\n\n\n

Phase 1: Spring 2023<\/h2>\n\n\n\n

We finalized the AirSim<\/strong> dataset collection<\/strong> pipeline for our rig setup as mentioned on the Setup<\/a> page. Then, we reviewed relevant literature<\/strong> and formulated two parallel approaches<\/strong> detailed on the Spring 2023<\/a> page. <\/p>\n\n\n\n

Presentation slides<\/a> and poster<\/a> are publicly available.<\/p>\n\n\n\n

Phase 2: Fall 2023<\/h2>\n\n\n\n

Due to the Sim2Real domain shift and minimal improvements with various convolution-based approaches, we switch to leveraging recent\u00a0foundation models<\/strong>, like DINOv2<\/strong>, for fisheye depth estimation<\/strong> and evaluate their performance on various environments and datasets which are detailed on the Fall 2023<\/a> page.<\/p>\n\n\n\n

Presentation slides<\/a> and poster<\/a> are publicly available.<\/p>\n","protected":false},"excerpt":{"rendered":"

Depth Estimation Holonomic robots such as drones have six degrees of freedom and thus, can move in all directions. Human drivers, too, rely on a wider field of view rather than just looking forward to making decisions. In the case of autonomous vehicles and self-driving cars, a LIDAR helps overcome this. However, LiDAR faces challenges … <\/p>\n

Continue reading “Introduction”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"","meta":{"footnotes":""},"class_list":["post-2","page","type-page","status-publish","hentry"],"yoast_head":"\nIntroduction - Multi-view robot perception on Edge Devices<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction - Multi-view robot perception on Edge Devices\" \/>\n<meta property=\"og:description\" content=\"Depth Estimation Holonomic robots such as drones have six degrees of freedom and thus, can move in all directions. Human drivers, too, rely on a wider field of view rather than just looking forward to making decisions. In the case of autonomous vehicles and self-driving cars, a LIDAR helps overcome this. However, LiDAR faces challenges … Continue reading "Introduction"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/\" \/>\n<meta property=\"og:site_name\" content=\"Multi-view robot perception on Edge Devices\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-18T22:03:47+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"name\":\"Introduction - Multi-view robot perception on Edge Devices\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#website\"},\"datePublished\":\"2023-05-01T17:48:32+00:00\",\"dateModified\":\"2023-12-18T22:03:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"name\":\"Multi-view robot perception on Edge Devices\",\"description\":\"CMU MSCV '23 capstone project\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#organization\",\"name\":\"Multi-view robot perception on Edge Devices\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/cropped-cropped-icon.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/wp-content\\\/uploads\\\/sites\\\/93\\\/2023\\\/05\\\/cropped-cropped-icon.png\",\"width\":250,\"height\":250,\"caption\":\"Multi-view robot perception on Edge Devices\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team16\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction - Multi-view robot perception on Edge Devices","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","og_locale":"en_US","og_type":"article","og_title":"Introduction - Multi-view robot perception on Edge Devices","og_description":"Depth Estimation Holonomic robots such as drones have six degrees of freedom and thus, can move in all directions. Human drivers, too, rely on a wider field of view rather than just looking forward to making decisions. In the case of autonomous vehicles and self-driving cars, a LIDAR helps overcome this. However, LiDAR faces challenges … Continue reading \"Introduction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","og_site_name":"Multi-view robot perception on Edge Devices","article_modified_time":"2023-12-18T22:03:47+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","name":"Introduction - Multi-view robot perception on Edge Devices","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#website"},"datePublished":"2023-05-01T17:48:32+00:00","dateModified":"2023-12-18T22:03:47+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/"},{"@type":"ListItem","position":2,"name":"Introduction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","name":"Multi-view robot perception on Edge Devices","description":"CMU MSCV '23 capstone project","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#organization","name":"Multi-view robot perception on Edge Devices","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/cropped-cropped-icon.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-content\/uploads\/sites\/93\/2023\/05\/cropped-cropped-icon.png","width":250,"height":250,"caption":"Multi-view robot perception on Edge Devices"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/2","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/comments?post=2"}],"version-history":[{"count":5,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/2\/revisions"}],"predecessor-version":[{"id":220,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/pages\/2\/revisions\/220"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team16\/wp-json\/wp\/v2\/media?parent=2"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}