{"id":194,"date":"2024-12-13T20:14:36","date_gmt":"2024-12-13T20:14:36","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/?page_id=194"},"modified":"2024-12-13T20:26:05","modified_gmt":"2024-12-13T20:26:05","slug":"3d-reconstruction","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/","title":{"rendered":"3D Reconstruction"},"content":{"rendered":"\n

Reconstruction of Dense 3D Point Cloud from Images<\/strong><\/p>\n\n\n\n

We aimed at lifting 2D images into 3D space for precise obstacle point clouds. The input consists of image sequences from the Roadwork Dataset. First, we feed the images into the Metric3D network to extract the metric depth of each image. Since the depth information can be inconsistent across frames, we then run Droid-SLAM to jointly optimize the camera poses and depth information. After these two steps, we can generate a point cloud like the one shown here. The pipeline is shown below.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Here\u2019s an example of our results. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Filtering<\/strong><\/p>\n\n\n\n

To get the positions and points of obstacles in road work areas, we process the 3D point cloud and validate points through projections across multiple 2D image masks. Points visible in a specific number of consecutive frames are retained as valid. Here is the demonstration of our SLAM-generated point cloud and our filtered obstacles. We see that by utilizing 2D segmentation labels, we successfully retain the points corresponding to obstacles while discarding irrelevant points. Below is some visualization.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Geo-alignment<\/strong><\/p>\n\n\n\n

We also apply a geo-alignment algorithm, so that the camera poses and point cloud are restored to the correct real-world scale. Figure (a) is The camera trajectories before geo-alignment. Figure (b) is The camera trajectories after geo-alignment.  The red trajectories represent the SLAM poses, while the blue trajectories depict the geo-aligned COLMAP poses.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Reconstruction of Dense 3D Point Cloud from Images We aimed at lifting 2D images into 3D space for precise obstacle point clouds. The input consists of image sequences from the Roadwork Dataset. First, we feed the images into the Metric3D network to extract the metric depth of each image. Since the depth information can be … <\/p>\n

Continue reading “3D Reconstruction”<\/span><\/a><\/p>\n","protected":false},"author":209,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-194","page","type-page","status-publish","hentry"],"yoast_head":"\n3D Reconstruction - Driving in Bad Weather<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"3D Reconstruction - Driving in Bad Weather\" \/>\n<meta property=\"og:description\" content=\"Reconstruction of Dense 3D Point Cloud from Images We aimed at lifting 2D images into 3D space for precise obstacle point clouds. The input consists of image sequences from the Roadwork Dataset. First, we feed the images into the Metric3D network to extract the metric depth of each image. Since the depth information can be … Continue reading "3D Reconstruction"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/\" \/>\n<meta property=\"og:site_name\" content=\"Driving in Bad Weather\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-13T20:26:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-content\/uploads\/sites\/107\/2024\/12\/3d-1024x213.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/\",\"name\":\"3D Reconstruction - Driving in Bad Weather\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/wp-content\\\/uploads\\\/sites\\\/107\\\/2024\\\/12\\\/3d-1024x213.png\",\"datePublished\":\"2024-12-13T20:14:36+00:00\",\"dateModified\":\"2024-12-13T20:26:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/wp-content\\\/uploads\\\/sites\\\/107\\\/2024\\\/12\\\/3d.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/wp-content\\\/uploads\\\/sites\\\/107\\\/2024\\\/12\\\/3d.png\",\"width\":1824,\"height\":380},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/3d-reconstruction\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"3D Reconstruction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/\",\"name\":\"Driving in Bad Weather\",\"description\":\"Students: Zi Wang, Peipei Zhong | Advisor: Srinivasa Narasimhan\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2024team9\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"3D Reconstruction - Driving in Bad Weather","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/","og_locale":"en_US","og_type":"article","og_title":"3D Reconstruction - Driving in Bad Weather","og_description":"Reconstruction of Dense 3D Point Cloud from Images We aimed at lifting 2D images into 3D space for precise obstacle point clouds. The input consists of image sequences from the Roadwork Dataset. First, we feed the images into the Metric3D network to extract the metric depth of each image. Since the depth information can be … Continue reading \"3D Reconstruction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/","og_site_name":"Driving in Bad Weather","article_modified_time":"2024-12-13T20:26:05+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-content\/uploads\/sites\/107\/2024\/12\/3d-1024x213.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/","name":"3D Reconstruction - Driving in Bad Weather","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-content\/uploads\/sites\/107\/2024\/12\/3d-1024x213.png","datePublished":"2024-12-13T20:14:36+00:00","dateModified":"2024-12-13T20:26:05+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-content\/uploads\/sites\/107\/2024\/12\/3d.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-content\/uploads\/sites\/107\/2024\/12\/3d.png","width":1824,"height":380},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/3d-reconstruction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/"},{"@type":"ListItem","position":2,"name":"3D Reconstruction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/","name":"Driving in Bad Weather","description":"Students: Zi Wang, Peipei Zhong | Advisor: Srinivasa Narasimhan","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/pages\/194","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/users\/209"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/comments?post=194"}],"version-history":[{"count":3,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/pages\/194\/revisions"}],"predecessor-version":[{"id":204,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/pages\/194\/revisions\/204"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2024team9\/wp-json\/wp\/v2\/media?parent=194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}