{"id":4,"date":"2023-05-07T02:02:14","date_gmt":"2023-05-07T02:02:14","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/?page_id=4"},"modified":"2023-12-15T21:33:16","modified_gmt":"2023-12-15T21:33:16","slug":"project-summary","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/","title":{"rendered":"About Our Project"},"content":{"rendered":"\n<p class=\"has-text-align-center\"><strong>Background<\/strong><\/p>\n\n\n\n<p class=\"has-text-align-center\">&nbsp;<img loading=\"lazy\" decoding=\"async\" width=\"972\" height=\"564\" class=\"wp-image-25\" style=\"width: 500px\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.02.07.png\" alt=\"Task\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.02.07.png 972w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.02.07-300x174.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.02.07-768x446.png 768w\" sizes=\"auto, (max-width: 972px) 100vw, 972px\" \/><\/p>\n\n\n\n<p>Autonomous vehicles are advancing rapidly, and a critical part of this progress is understanding the environment around them. For these vehicles to be truly effective and safe, they must be able to recognize not only standard objects that are often annotated during training, such as other vehicles or pedestrians, but also identify and communicate about unusual or unexpected items on the road. This could be anything from construction materials to an animal.&nbsp;<\/p>\n\n\n\n<p>In the joint exploration between Honda 99P Labs and CMU MSCV capstone project, our overarching goal is to enhance the reliability of self-driving cars. By having a detailed and accurate map of both expected and unexpected objects, we hope the cars can gain a thorough understanding of their environment. By accurately classifying and describing these objects, the autonomous system can make informed decisions, ensuring it reacts correctly to different situations. This comprehensive scene understanding is fundamental for preventing accidents and ensuring a safer journey for passengers and those around the vehicle.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Project Goals:&nbsp;<\/strong>\n<ul class=\"wp-block-list\">\n<li>1. Identifying Road Conditions(<strong>RC<\/strong>) via semantic understanding\n<ul class=\"wp-block-list\">\n<li>Parsing expected and unexpected\/rare RCs, objects, and layouts<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>2. Identifying Rare RCs and Unknown\/rare Objects\n<ul class=\"wp-block-list\">\n<li>Discovering objects and assessing RCs by open-set models<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Task Formulation:<\/strong>\n<ul class=\"wp-block-list\">\n<li>Frame-wise RC identification and localization by point-level semantic segmentation and object-level detection<\/li>\n\n\n\n<li>Improving the robustness of known points and objects<\/li>\n\n\n\n<li>Parsing unknown points and objects<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"970\" height=\"754\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.06.54-1.png\" alt=\"\" class=\"wp-image-27\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.06.54-1.png 970w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.06.54-1-300x233.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.06.54-1-768x597.png 768w\" sizes=\"auto, (max-width: 970px) 100vw, 970px\" \/><\/figure>\n\n\n\n<p>Baseline Network: <\/p>\n\n\n\n<p><em>2DPASS<\/em>: <em>3D Segmentation on LiDAR Point Clouds Assisted by 2D Priors, ECCV, 2022<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"324\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16-1024x324.png\" alt=\"\" class=\"wp-image-28\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16-1024x324.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16-300x95.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16-768x243.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16-1536x486.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-content\/uploads\/sites\/79\/2023\/05\/\u622a\u5716-2023-05-07-\u4e0b\u534811.10.16.png 1542w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Background &nbsp; Autonomous vehicles are advancing rapidly, and a critical part of this progress is understanding the environment around them. For these vehicles to be truly effective and safe, they must be able to recognize not only standard objects that are often annotated during training, such as other vehicles or pedestrians, but also identify and [&hellip;]<\/p>\n","protected":false},"author":157,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-4","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>About Our Project - Semantic Understanding of Road Scenes<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"About Our Project - Semantic Understanding of Road Scenes\" \/>\n<meta property=\"og:description\" content=\"Background &nbsp; Autonomous vehicles are advancing rapidly, and a critical part of this progress is understanding the environment around them. For these vehicles to be truly effective and safe, they must be able to recognize not only standard objects that are often annotated during training, such as other vehicles or pedestrians, but also identify and [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/\" \/>\n<meta property=\"og:site_name\" content=\"Semantic Understanding of Road Scenes\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-15T21:33:16+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/project-summary\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/project-summary\\\/\",\"name\":\"About Our Project - Semantic Understanding of Road Scenes\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/#website\"},\"datePublished\":\"2023-05-07T02:02:14+00:00\",\"dateModified\":\"2023-12-15T21:33:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/project-summary\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/project-summary\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/project-summary\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"About Our Project\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/\",\"name\":\"Semantic Understanding of Road Scenes\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team2\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"About Our Project - Semantic Understanding of Road Scenes","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/","og_locale":"en_US","og_type":"article","og_title":"About Our Project - Semantic Understanding of Road Scenes","og_description":"Background &nbsp; Autonomous vehicles are advancing rapidly, and a critical part of this progress is understanding the environment around them. For these vehicles to be truly effective and safe, they must be able to recognize not only standard objects that are often annotated during training, such as other vehicles or pedestrians, but also identify and [&hellip;]","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/","og_site_name":"Semantic Understanding of Road Scenes","article_modified_time":"2023-12-15T21:33:16+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/","name":"About Our Project - Semantic Understanding of Road Scenes","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/#website"},"datePublished":"2023-05-07T02:02:14+00:00","dateModified":"2023-12-15T21:33:16+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/project-summary\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/"},{"@type":"ListItem","position":2,"name":"About Our Project"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/","name":"Semantic Understanding of Road Scenes","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/pages\/4","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/users\/157"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/comments?post=4"}],"version-history":[{"count":4,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/pages\/4\/revisions"}],"predecessor-version":[{"id":87,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/pages\/4\/revisions\/87"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team2\/wp-json\/wp\/v2\/media?parent=4"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}