{"id":138,"date":"2023-12-18T22:22:51","date_gmt":"2023-12-18T22:22:51","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/?p=138"},"modified":"2023-12-18T22:40:44","modified_gmt":"2023-12-18T22:40:44","slug":"annotations","status":"publish","type":"post","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/","title":{"rendered":"Annotations"},"content":{"rendered":"\n<h1 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:600\">Human-level Annotations<\/h1>\n\n\n\n<p>We include 2D and 3D pose estimation as part of our data collection pipeline. Human-pose can be useful to obtain a trajectory and interaction information which is a good prior for imitation frameworks.<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"400\" style=\"aspect-ratio: 800 \/ 400;\" width=\"800\" controls src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/05\/vis_kitchen_2.mp4\"><\/video><\/figure>\n\n\n\n<p>We also include hand-pose estimation and segmentation.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"489\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png\" alt=\"\" class=\"wp-image-140\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-300x143.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-768x366.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1536x733.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-2048x977.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:600\">Scene-level Annotations<\/h1>\n\n\n\n<p>Our data contains posed RGB-D images that can provide us with <em>point clouds<\/em>. Therefore, for scene-level annotations, we have focused on collecting 3D segmentation masks for <em>point clouds<\/em>. <\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:600\">Leveraging 2D Foundation Models For 3D Scene Segmentation<\/h3>\n\n\n\n<p>Modern 2D segmentation foundation models, enabled by transfer learning and large-scale datasets generate high-quality object masks. 3D segmentation models are not there yet. Given the fact that a majority of 3D data are collected by sensors that produce point clouds from RGB-D images, we aim to leverage the 2D foundation models that take RGB images as input to yield segmentation masks and develop an algorithm that utilizes those generated masks along with the depth information and scene geometry to facilitate 3D scene segmentation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:500\">Finding Correspondences<\/h3>\n\n\n\n<p>Each view has its corresponding color images and depth images, from which we can get a point cloud and 2D-to-3D correspondences. Using camera matrix of the next view, we can find 2D-to-2D correspondences between the current view and the next view. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"462\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-1024x462.png\" alt=\"\" class=\"wp-image-144\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-1024x462.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-300x135.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-768x347.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-1536x693.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_a-2048x924.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:500\">Graph-Based Merging<\/h3>\n\n\n\n<p>The unsupervised foundation models do not provide semantic IDs for the segmentation generated. Moreover, due to occlusion, objects might be separated into two masks.<\/p>\n\n\n\n<p>To determine the merging assignments of masks across views, we formulate a graph as shown in the diagram below. Based on 2D-to-2D correspondences, we decide the edges between two nodes. After updating all the masks in all the views, we eventually reach to a final graph and all the connected nodes in the final graph are assigned same ID. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"636\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b-1024x636.png\" alt=\"\" class=\"wp-image-145\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b-1024x636.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b-300x186.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b-768x477.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b-1536x954.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/method_b.png 1965w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading has-text-align-center\" style=\"font-style:normal;font-weight:600\">Results<\/h1>\n\n\n\n<h4 class=\"wp-block-heading\">Before merging:<\/h4>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min-1024x768.png\" alt=\"\" class=\"wp-image-147\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min-1024x768.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min-300x225.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min-768x576.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min-1536x1152.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_0-min.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min-1024x768.png\" alt=\"\" class=\"wp-image-148\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min-1024x768.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min-300x225.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min-768x576.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min-1536x1152.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/masks_1080_2-min.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<h4 class=\"wp-block-heading\">After merging:<\/h4>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min-1024x768.png\" alt=\"\" class=\"wp-image-149\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min-1024x768.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min-300x225.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min-768x576.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min-1536x1152.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_0-min.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min-1024x768.png\" alt=\"\" class=\"wp-image-150\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min-1024x768.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min-300x225.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min-768x576.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min-1536x1152.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged_masks_1080_2-min.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<h4 class=\"wp-block-heading\">Output merged point cloud:<\/h4>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"841\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged-pcd-1024x841.png\" alt=\"\" class=\"wp-image-151\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged-pcd-1024x841.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged-pcd-300x246.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged-pcd-768x630.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/merged-pcd.png 1256w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Human-level Annotations We include 2D and 3D pose estimation as part of our data collection pipeline. Human-pose can be useful to obtain a trajectory and interaction information which is a good prior for imitation frameworks. We also include hand-pose estimation and segmentation. Scene-level Annotations Our data contains posed RGB-D images that can provide us with &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Annotations&#8221;<\/span><\/a><\/p>\n","protected":false},"author":169,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-138","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Annotations - 3D Kitchen Understanding<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Annotations - 3D Kitchen Understanding\" \/>\n<meta property=\"og:description\" content=\"Human-level Annotations We include 2D and 3D pose estimation as part of our data collection pipeline. Human-pose can be useful to obtain a trajectory and interaction information which is a good prior for imitation frameworks. We also include hand-pose estimation and segmentation. Scene-level Annotations Our data contains posed RGB-D images that can provide us with &hellip; Continue reading &quot;Annotations&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/\" \/>\n<meta property=\"og:site_name\" content=\"3D Kitchen Understanding\" \/>\n<meta property=\"article:published_time\" content=\"2023-12-18T22:22:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-18T22:40:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png\" \/>\n<meta name=\"author\" content=\"achleshl\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"achleshl\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/\"},\"author\":{\"name\":\"achleshl\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/#\\\/schema\\\/person\\\/6365ce88474ebf4013760ad402320da9\"},\"headline\":\"Annotations\",\"datePublished\":\"2023-12-18T22:22:51+00:00\",\"dateModified\":\"2023-12-18T22:40:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/\"},\"wordCount\":304,\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/wp-content\\\/uploads\\\/sites\\\/85\\\/2023\\\/12\\\/hand-annotations-1024x489.png\",\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/\",\"name\":\"Annotations - 3D Kitchen Understanding\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/wp-content\\\/uploads\\\/sites\\\/85\\\/2023\\\/12\\\/hand-annotations-1024x489.png\",\"datePublished\":\"2023-12-18T22:22:51+00:00\",\"dateModified\":\"2023-12-18T22:40:44+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/#\\\/schema\\\/person\\\/6365ce88474ebf4013760ad402320da9\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/wp-content\\\/uploads\\\/sites\\\/85\\\/2023\\\/12\\\/hand-annotations.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/wp-content\\\/uploads\\\/sites\\\/85\\\/2023\\\/12\\\/hand-annotations.png\",\"width\":2100,\"height\":1002},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/2023\\\/12\\\/18\\\/annotations\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Annotations\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/\",\"name\":\"3D Kitchen Understanding\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/#\\\/schema\\\/person\\\/6365ce88474ebf4013760ad402320da9\",\"name\":\"achleshl\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g\",\"caption\":\"achleshl\"},\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team8\\\/author\\\/achleshl\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Annotations - 3D Kitchen Understanding","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/","og_locale":"en_US","og_type":"article","og_title":"Annotations - 3D Kitchen Understanding","og_description":"Human-level Annotations We include 2D and 3D pose estimation as part of our data collection pipeline. Human-pose can be useful to obtain a trajectory and interaction information which is a good prior for imitation frameworks. We also include hand-pose estimation and segmentation. Scene-level Annotations Our data contains posed RGB-D images that can provide us with &hellip; Continue reading \"Annotations\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/","og_site_name":"3D Kitchen Understanding","article_published_time":"2023-12-18T22:22:51+00:00","article_modified_time":"2023-12-18T22:40:44+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png","type":"","width":"","height":""}],"author":"achleshl","twitter_card":"summary_large_image","twitter_misc":{"Written by":"achleshl","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#article","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/"},"author":{"name":"achleshl","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/#\/schema\/person\/6365ce88474ebf4013760ad402320da9"},"headline":"Annotations","datePublished":"2023-12-18T22:22:51+00:00","dateModified":"2023-12-18T22:40:44+00:00","mainEntityOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/"},"wordCount":304,"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png","inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/","name":"Annotations - 3D Kitchen Understanding","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations-1024x489.png","datePublished":"2023-12-18T22:22:51+00:00","dateModified":"2023-12-18T22:40:44+00:00","author":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/#\/schema\/person\/6365ce88474ebf4013760ad402320da9"},"breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-content\/uploads\/sites\/85\/2023\/12\/hand-annotations.png","width":2100,"height":1002},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/12\/18\/annotations\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/"},{"@type":"ListItem","position":2,"name":"Annotations"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/","name":"3D Kitchen Understanding","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/#\/schema\/person\/6365ce88474ebf4013760ad402320da9","name":"achleshl","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6687ebf50bf21ae5f186b3d766400f03f98c316465dafc024b52ddad21594407?s=96&d=mm&r=g","caption":"achleshl"},"url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/author\/achleshl\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/posts\/138","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/users\/169"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/comments?post=138"}],"version-history":[{"count":4,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/posts\/138\/revisions"}],"predecessor-version":[{"id":152,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/posts\/138\/revisions\/152"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/media?parent=138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/categories?post=138"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/wp-json\/wp\/v2\/tags?post=138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}