{"id":95,"date":"2022-04-29T16:37:07","date_gmt":"2022-04-29T16:37:07","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/?page_id=95"},"modified":"2022-04-30T14:46:59","modified_gmt":"2022-04-30T14:46:59","slug":"experiments","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/","title":{"rendered":"Experiments"},"content":{"rendered":"\n

Datasets<\/h2>\n\n\n\n

Traditional Stereo Datasets<\/strong>
Due to the difficulty of getting labeled ground truth depth estimates, there are only a limited number of datasets. The most popular ones are KITTI, Middlebury, and ETH3D. However, these datasets are almost entirely focused on daylight with very few low-light environmental conditions.<\/p>\n\n\n\n

Oxford Robot Car<\/strong>
Contains a large number of outdoor driving around Oxford at day and night time, as well as various weather conditions.<\/p>\n\n\n\n

\n
\n

NREC Collected Data<\/strong>
The above datasets, while may be helpful to some extent, are certainly not in the same domain as the environments for our targeted application. Mainly, our environment contains not just low-light but also off-road environments for which there aren\u2019t any publicly available datasets yet. As such, NREC has collected their own data at various locations that are closer to our actual domain. An example of one such locations during day-time is shown.<\/p>\n<\/div>\n\n\n\n

\n
\"\"
Sample Day RGB Image at an NREC Test Site<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Results<\/h2>\n\n\n\n

Evaluation Metrics<\/strong>
We use standard evaluation metrics listed below from KITTI and Middlebury datasets.<\/p>\n\n\n\n

  1. Average Error<\/em> – The average error between ground truth disparity and predicted disparity over the entire image and dataset.<\/li>
  2. Bad-X Error<\/em> – The percentage of pixels in the entire image which have an error > X in disparity.<\/li><\/ol>\n\n\n\n

    We trained and evaluated Deep HSM on the KITTI dataset. The Deep HSM model was initially trained on a large synthetic dataset, and then fine-tuned on KITTI. Below are the results.<\/p>\n\n\n\n

    Distance (meters)<\/strong><\/td>Average Error<\/strong><\/td>Bad-1<\/strong> %<\/strong><\/td>Bad-4 %<\/strong><\/td><\/tr>
    0-25<\/td>1.245<\/td>27.1<\/td>3.5<\/td><\/tr>
    25-60<\/td>1.147<\/td>27.7<\/td>3.9<\/td><\/tr>
    60-115<\/td>1.196<\/td>39.0<\/td>4.3<\/td><\/tr>
    0-115<\/strong><\/td>1.231<\/td>30.4<\/td>4.1<\/td><\/tr><\/tbody><\/table>
    Evaluation Metrics on KITTI w\/ Deep HSM<\/figcaption><\/figure>\n\n\n\n

    Visualization Results<\/strong><\/p>\n\n\n\n

    We showcase the following:<\/p>\n\n\n\n

    \n
    \n

    Input Image<\/strong><\/p>\n<\/div>\n\n\n\n

    \n

    Predicted Disparity Map<\/strong><\/p>\n<\/div>\n<\/div>\n\n\n\n

    \n
    \n

    <\/p>\n<\/div>\n\n\n\n

    \n

    <\/p>\n<\/div>\n<\/div>\n\n\n\n

    \n
    \n

    <\/p>\n<\/div>\n\n\n\n

    \n

    <\/p>\n<\/div>\n<\/div>\n\n\n\n

    \n
    \n

    <\/p>\n<\/div>\n\n\n\n

    \n

    <\/p>\n<\/div>\n<\/div>\n\n\n\n

    Entropy<\/strong>
    We observe that edges of objects have the highest uncertainty as well as overall there is higher entropy in darker images as shown in the examples below.<\/p>\n\n\n\n

    \n
    \n

    <\/p>\n<\/div>\n\n\n\n

    \n

    <\/p>\n<\/div>\n<\/div>\n\n\n\n

    \n
    \n

    <\/p>\n<\/div>\n\n\n\n

    \n

    <\/p>\n<\/div>\n<\/div>\n\n\n\n

    <\/p>\n","protected":false},"excerpt":{"rendered":"

    Datasets Traditional Stereo DatasetsDue to the difficulty of getting labeled ground truth depth estimates, there are only a limited number of datasets. The most popular ones are KITTI, Middlebury, and ETH3D. However, these datasets are almost entirely focused on daylight with very few low-light environmental conditions. Oxford Robot CarContains a large number of outdoor driving … <\/p>\n

    Continue reading “Experiments”<\/span><\/a><\/p>\n","protected":false},"author":121,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-95","page","type-page","status-publish","hentry"],"yoast_head":"\nExperiments - Depth Estimation in Low-light Environments for Autonomous Navigation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Experiments - Depth Estimation in Low-light Environments for Autonomous Navigation\" \/>\n<meta property=\"og:description\" content=\"Datasets Traditional Stereo DatasetsDue to the difficulty of getting labeled ground truth depth estimates, there are only a limited number of datasets. The most popular ones are KITTI, Middlebury, and ETH3D. However, these datasets are almost entirely focused on daylight with very few low-light environmental conditions. Oxford Robot CarContains a large number of outdoor driving … Continue reading "Experiments"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/\" \/>\n<meta property=\"og:site_name\" content=\"Depth Estimation in Low-light Environments for Autonomous Navigation\" \/>\n<meta property=\"article:modified_time\" content=\"2022-04-30T14:46:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/04\/p0m2l2_13166_1581449430438801-edited-scaled.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/\",\"name\":\"Experiments - Depth Estimation in Low-light Environments for Autonomous Navigation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/wp-content\\\/uploads\\\/sites\\\/62\\\/2022\\\/04\\\/p0m2l2_13166_1581449430438801-edited-scaled.jpg\",\"datePublished\":\"2022-04-29T16:37:07+00:00\",\"dateModified\":\"2022-04-30T14:46:59+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/wp-content\\\/uploads\\\/sites\\\/62\\\/2022\\\/04\\\/p0m2l2_13166_1581449430438801-edited-scaled.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/wp-content\\\/uploads\\\/sites\\\/62\\\/2022\\\/04\\\/p0m2l2_13166_1581449430438801-edited-scaled.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/experiments\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Experiments\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/\",\"name\":\"Depth Estimation in Low-light Environments for Autonomous Navigation\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#organization\",\"name\":\"Carnegie Mellon University\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/wp-content\\\/uploads\\\/sites\\\/62\\\/2022\\\/12\\\/cmu-wordmark-stacked-r.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/wp-content\\\/uploads\\\/sites\\\/62\\\/2022\\\/12\\\/cmu-wordmark-stacked-r.png\",\"width\":301,\"height\":193,\"caption\":\"Carnegie Mellon University\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team7\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Experiments - Depth Estimation in Low-light Environments for Autonomous Navigation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/","og_locale":"en_US","og_type":"article","og_title":"Experiments - Depth Estimation in Low-light Environments for Autonomous Navigation","og_description":"Datasets Traditional Stereo DatasetsDue to the difficulty of getting labeled ground truth depth estimates, there are only a limited number of datasets. The most popular ones are KITTI, Middlebury, and ETH3D. However, these datasets are almost entirely focused on daylight with very few low-light environmental conditions. Oxford Robot CarContains a large number of outdoor driving … Continue reading \"Experiments\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/","og_site_name":"Depth Estimation in Low-light Environments for Autonomous Navigation","article_modified_time":"2022-04-30T14:46:59+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/04\/p0m2l2_13166_1581449430438801-edited-scaled.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/","name":"Experiments - Depth Estimation in Low-light Environments for Autonomous Navigation","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/04\/p0m2l2_13166_1581449430438801-edited-scaled.jpg","datePublished":"2022-04-29T16:37:07+00:00","dateModified":"2022-04-30T14:46:59+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/04\/p0m2l2_13166_1581449430438801-edited-scaled.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/04\/p0m2l2_13166_1581449430438801-edited-scaled.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/experiments\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/"},{"@type":"ListItem","position":2,"name":"Experiments"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/","name":"Depth Estimation in Low-light Environments for Autonomous Navigation","description":"","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#organization","name":"Carnegie Mellon University","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/12\/cmu-wordmark-stacked-r.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-content\/uploads\/sites\/62\/2022\/12\/cmu-wordmark-stacked-r.png","width":301,"height":193,"caption":"Carnegie Mellon University"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/pages\/95","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/users\/121"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/comments?post=95"}],"version-history":[{"count":13,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/pages\/95\/revisions"}],"predecessor-version":[{"id":153,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/pages\/95\/revisions\/153"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team7\/wp-json\/wp\/v2\/media?parent=95"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}