{"id":108,"date":"2023-05-09T00:36:43","date_gmt":"2023-05-09T00:36:43","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/?page_id=108"},"modified":"2023-12-18T07:18:13","modified_gmt":"2023-12-18T07:18:13","slug":"related-work","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/","title":{"rendered":"Related work"},"content":{"rendered":"\n

PointPillars: <\/strong><\/h1>\n\n\n\n


PointPillars is a pioneering object detection system specifically designed for efficiently detecting and localizing objects in 3D point clouds, typically obtained from LiDAR sensors. It operates by transforming the irregularly spaced point cloud data into a structured format to enable precise object recognition. The approach utilizes a two-stage architecture, where the first stage generates preliminary voxel-based representations of the point cloud, and the second stage refines these representations to predict object attributes like position, size, and orientation. Its key strength lies in its ability to achieve high accuracy in object detection while maintaining computational efficiency, making it well-suited for real-time applications in autonomous vehicles and robotics.<\/p>\n\n\n\n

One notable aspect of PointPillars is its focus on optimizing both accuracy and speed in 3D object detection. By leveraging a sparse convolutional neural network (CNN) architecture, PointPillars efficiently processes the volumetric data obtained from LiDAR sensors. This approach significantly reduces computational costs compared to traditional dense CNNs, enabling faster inference times without compromising accuracy. Additionally, PointPillars incorporates a novel set of anchor-free prediction heads, allowing it to precisely locate objects in the point cloud without relying on predefined anchor boxes, contributing to its adaptability and robustness across diverse environments.<\/p>\n\n\n\n

The effectiveness of PointPillars lies in its ability to handle complex real-world scenarios, including varying object sizes, occlusions, and cluttered environments. Its robustness and accuracy in detecting objects like vehicles, pedestrians, and cyclists in challenging conditions make it a valuable tool for enhancing the perception capabilities of autonomous vehicles, enabling safer navigation and decision-making. Despite its advancements, ongoing research and development efforts continue to refine PointPillars and address challenges related to scalability, further optimizing its performance across different scenarios and sensor configurations.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

SOLOv2<\/strong><\/h1>\n\n\n\n

SOLOv2 is a simple, direct, and fast framework for instance segmentation in 2D images. It is empowered by an efficient and holistic instance mask representation scheme, which
dynamically segments each instance in the image, without resorting to bounding
box detection. SOLOv2 significantly reduces the inference time with a novel matrix non-maximum suppression (NMS) technique. The authors propose to learn adaptive and dynamic convolutional kernels for the mask prediction, leading to a much more compact and powerful design, which yields better results.

We propose to use the SOLOv2 architecture for segmenting 2D range image representation of our point clouds. This will ensure that the 3D point cloud segmentation can be done in real time.<\/p>\n\n\n\n

\"\"
SOLOv2 architecture for real time 2D instance segmentation<\/figcaption><\/figure>\n\n\n\n

References<\/span><\/strong>

[1] Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In CVPR, 2019.

[2] Wang, Xinlong, Rufeng Zhang, Tao Kong, Lei Li, and Chunhua Shen. “Solov2: Dynamic and fast instance segmentation.”\u00a0Advances in Neural information processing systems<\/em>\u00a033 (2020): 17721-17732.<\/p>\n","protected":false},"excerpt":{"rendered":"

PointPillars: PointPillars is a pioneering object detection system specifically designed for efficiently detecting and localizing objects in 3D point clouds, typically obtained from LiDAR sensors. It operates by transforming the irregularly spaced point cloud data into a structured format to enable precise object recognition. The approach utilizes a two-stage architecture, where the first stage generates … <\/p>\n

Continue reading “Related work”<\/span><\/a><\/p>\n","protected":false},"author":185,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-108","page","type-page","status-publish","hentry"],"yoast_head":"\nRelated work - Semantic 3D MAPPING<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Related work - Semantic 3D MAPPING\" \/>\n<meta property=\"og:description\" content=\"PointPillars: PointPillars is a pioneering object detection system specifically designed for efficiently detecting and localizing objects in 3D point clouds, typically obtained from LiDAR sensors. It operates by transforming the irregularly spaced point cloud data into a structured format to enable precise object recognition. The approach utilizes a two-stage architecture, where the first stage generates … Continue reading "Related work"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/\" \/>\n<meta property=\"og:site_name\" content=\"Semantic 3D MAPPING\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-18T07:18:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-content\/uploads\/sites\/94\/2023\/12\/image-1-1024x330.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/\",\"name\":\"Related work - Semantic 3D MAPPING\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/wp-content\\\/uploads\\\/sites\\\/94\\\/2023\\\/12\\\/image-1-1024x330.png\",\"datePublished\":\"2023-05-09T00:36:43+00:00\",\"dateModified\":\"2023-12-18T07:18:13+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/wp-content\\\/uploads\\\/sites\\\/94\\\/2023\\\/12\\\/image-1.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/wp-content\\\/uploads\\\/sites\\\/94\\\/2023\\\/12\\\/image-1.png\",\"width\":1364,\"height\":440},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/related-work\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Related work\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/\",\"name\":\"Semantic 3D MAPPING\",\"description\":\"Real time instance level semantic segmentation of 3D point clouds in indoor environments.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team17\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Related work - Semantic 3D MAPPING","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/","og_locale":"en_US","og_type":"article","og_title":"Related work - Semantic 3D MAPPING","og_description":"PointPillars: PointPillars is a pioneering object detection system specifically designed for efficiently detecting and localizing objects in 3D point clouds, typically obtained from LiDAR sensors. It operates by transforming the irregularly spaced point cloud data into a structured format to enable precise object recognition. The approach utilizes a two-stage architecture, where the first stage generates … Continue reading \"Related work\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/","og_site_name":"Semantic 3D MAPPING","article_modified_time":"2023-12-18T07:18:13+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-content\/uploads\/sites\/94\/2023\/12\/image-1-1024x330.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/","name":"Related work - Semantic 3D MAPPING","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-content\/uploads\/sites\/94\/2023\/12\/image-1-1024x330.png","datePublished":"2023-05-09T00:36:43+00:00","dateModified":"2023-12-18T07:18:13+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-content\/uploads\/sites\/94\/2023\/12\/image-1.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-content\/uploads\/sites\/94\/2023\/12\/image-1.png","width":1364,"height":440},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/related-work\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/"},{"@type":"ListItem","position":2,"name":"Related work"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/","name":"Semantic 3D MAPPING","description":"Real time instance level semantic segmentation of 3D point clouds in indoor environments.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/pages\/108","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/users\/185"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/comments?post=108"}],"version-history":[{"count":9,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/pages\/108\/revisions"}],"predecessor-version":[{"id":160,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/pages\/108\/revisions\/160"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team17\/wp-json\/wp\/v2\/media?parent=108"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}