{"id":126,"date":"2020-12-15T23:20:14","date_gmt":"2020-12-16T04:20:14","guid":{"rendered":"http:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/?page_id=126"},"modified":"2020-12-17T00:49:46","modified_gmt":"2020-12-17T05:49:46","slug":"fall-semester","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/","title":{"rendered":"Fall 2020"},"content":{"rendered":"\n<p>In the Spring semester, we conclude that if we have perfect 2D image segmentation, that could benefit the 3D object detection model by a large margin. Given this finding, we propose to improve 2D image segmentation given 3D object detection result. If that is also true, we could close the loop and let both tasks benefit each other. Here is the illustrative figure:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"533\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-1024x533.jpg\" alt=\"\" class=\"wp-image-153\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-1024x533.jpg 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-300x156.jpg 300w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-768x400.jpg 768w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-1536x800.jpg 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-2048x1067.jpg 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p>We propose to address two issues simultaneously: (1) fuse 3D information into 2D segmentation (2) sequential inference by cascading the detection model. We highlight the main process in the following two figures:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"530\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-1024x530.jpg\" alt=\"\" class=\"wp-image-155\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-1024x530.jpg 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-300x155.jpg 300w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-768x398.jpg 768w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-1536x796.jpg 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im3-1-2048x1061.jpg 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"532\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-1024x532.jpg\" alt=\"\" class=\"wp-image-154\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-1024x532.jpg 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-300x156.jpg 300w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-768x399.jpg 768w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-1536x798.jpg 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im2-1-2048x1064.jpg 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p>For details, please refer to our presentation <a href=\"https:\/\/drive.google.com\/file\/d\/1F14PjnBW-Vt5ijpa-6Pc7ekpjXE0EURq\/view?usp=sharing\">video<\/a> and <a href=\"https:\/\/drive.google.com\/file\/d\/1LuhpICsb93c9Mr9y3Uq-zs27wnm2Ssgy\/view?usp=sharing\">slides<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the Spring semester, we conclude that if we have perfect 2D image segmentation, that could benefit the 3D object detection model by a large margin. Given this finding, we propose to improve 2D image segmentation given 3D object detection result. If that is also true, we could close the loop and let both tasks &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Fall 2020&#8221;<\/span><\/a><\/p>\n","protected":false},"author":81,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-126","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fall 2020 - Multimodal 3D Object Detection<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fall 2020 - Multimodal 3D Object Detection\" \/>\n<meta property=\"og:description\" content=\"In the Spring semester, we conclude that if we have perfect 2D image segmentation, that could benefit the 3D object detection model by a large margin. Given this finding, we propose to improve 2D image segmentation given 3D object detection result. If that is also true, we could close the loop and let both tasks &hellip; Continue reading &quot;Fall 2020&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/\" \/>\n<meta property=\"og:site_name\" content=\"Multimodal 3D Object Detection\" \/>\n<meta property=\"article:modified_time\" content=\"2020-12-17T05:49:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2050\" \/>\n\t<meta property=\"og:image:height\" content=\"1068\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/\",\"name\":\"Fall 2020 - Multimodal 3D Object Detection\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/wp-content\\\/uploads\\\/sites\\\/36\\\/2020\\\/12\\\/im1-1-1024x533.jpg\",\"datePublished\":\"2020-12-16T04:20:14+00:00\",\"dateModified\":\"2020-12-17T05:49:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/wp-content\\\/uploads\\\/sites\\\/36\\\/2020\\\/12\\\/im1-1.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/wp-content\\\/uploads\\\/sites\\\/36\\\/2020\\\/12\\\/im1-1.jpg\",\"width\":2050,\"height\":1068},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/fall-semester\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fall 2020\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/\",\"name\":\"Multimodal 3D Object Detection\",\"description\":\"Advised by Kris Kitani and Varun Ramakrishna | Sponsored by Aurora Innovation\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamf\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fall 2020 - Multimodal 3D Object Detection","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/","og_locale":"en_US","og_type":"article","og_title":"Fall 2020 - Multimodal 3D Object Detection","og_description":"In the Spring semester, we conclude that if we have perfect 2D image segmentation, that could benefit the 3D object detection model by a large margin. Given this finding, we propose to improve 2D image segmentation given 3D object detection result. If that is also true, we could close the loop and let both tasks &hellip; Continue reading \"Fall 2020\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/","og_site_name":"Multimodal 3D Object Detection","article_modified_time":"2020-12-17T05:49:46+00:00","og_image":[{"width":2050,"height":1068,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/","name":"Fall 2020 - Multimodal 3D Object Detection","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1-1024x533.jpg","datePublished":"2020-12-16T04:20:14+00:00","dateModified":"2020-12-17T05:49:46+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-content\/uploads\/sites\/36\/2020\/12\/im1-1.jpg","width":2050,"height":1068},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/fall-semester\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/"},{"@type":"ListItem","position":2,"name":"Fall 2020"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/","name":"Multimodal 3D Object Detection","description":"Advised by Kris Kitani and Varun Ramakrishna | Sponsored by Aurora Innovation","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/pages\/126","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/users\/81"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/comments?post=126"}],"version-history":[{"count":6,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/pages\/126\/revisions"}],"predecessor-version":[{"id":161,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/pages\/126\/revisions\/161"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamf\/wp-json\/wp\/v2\/media?parent=126"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}