{"id":8,"date":"2025-05-06T03:06:49","date_gmt":"2025-05-06T03:06:49","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/?page_id=8"},"modified":"2025-12-13T03:23:38","modified_gmt":"2025-12-13T03:23:38","slug":"overview","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","title":{"rendered":"\u200b"},"content":{"rendered":"\n

Overview<\/strong><\/h1>\n\n\n\n

Motivation<\/strong><\/h2>\n\n\n\n

Robots hold immense promise in transforming society\u2014supporting disaster response assisting in eldercare, performing hazardous inspections, and enhancing productivity across industries. Yet despite recent hardware advancements, robot learning remains far behind the progress seen in computer vision and natural language processing. While vision and language models benefit from massive web-scale datasets, robots still rely heavily on simulation or narrowly curated demonstrations. As a result, they struggle to generalize to the real world.<\/p>\n\n\n\n

Problem Statement<\/strong><\/h2>\n\n\n\n

Despite improvements in perception and control, robot learning faces three major challenges: sparse rewards in reinforcement learning<\/strong>, limited access to diverse motion data<\/strong>, and a fundamental embodiment gap<\/strong> between animals and machines. These barriers prevent current methods from scaling and generalizing effectively.<\/p>\n\n\n\n

    \n
  1. Sparse Rewards in Reinforcement Learning<\/strong>
    Sparse rewards in reinforcement learning makes it difficult for robots to efficiently acquire complex behaviors. This is because success is observed only after long, complex trajectories. The delay in providing reward feedback makes training slow and learning hard.<\/li>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
  2. Limited Robot Learning Datasets<\/strong>
    Most robot learning datasets are small, domain-specific, and costly to collect. In contrast, computer vision benefits from abundant internet-scale data. The lack of large scale, diverse datasets hampers the ability of robots to learn from a wide variety of scenarios. <\/li>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
  3. Embodiment Gap<\/strong>
    Even for robots inspired by real creatures, a significant embodiment gap remains between the robot and its biological counterpart. Biological creatures exhibit flexible, high-degree-of-freedom motion, while robots are constrained by their structure and actuators. This mismatch makes it difficult to directly imitate behaviors from human and animal videos.<\/li>\n\n\n\n
    \"\"<\/figure>\n<\/ol>\n\n\n\n

    Our Goal<\/strong><\/h2>\n\n\n\n

    To address these challenges, our project aims to leverage reconstruction to help robots learn from large-scale, in-the-wild video data by extracting motion representations that are both visually grounded and compatible with robotic embodiments. By creating a framework that leverages visual data from the real world to generate transferrable robot motions, we aim to enable scalable learning of diverse behaviors grounded in everyday human and animal motion.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Overview Motivation Robots hold immense promise in transforming society\u2014supporting disaster response assisting in eldercare, performing hazardous inspections, and enhancing productivity across industries. Yet despite recent hardware advancements, robot learning remains far behind the progress seen in computer vision and natural language processing. While vision and language models benefit from massive web-scale datasets, robots still rely … <\/p>\n

    Continue reading “\u200b”<\/span><\/a><\/p>\n","protected":false},"author":228,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-8","page","type-page","status-publish","hentry"],"yoast_head":"\n\u200b - 4D reconstruction From Monocular Videos<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u200b - 4D reconstruction From Monocular Videos\" \/>\n<meta property=\"og:description\" content=\"Overview Motivation Robots hold immense promise in transforming society\u2014supporting disaster response assisting in eldercare, performing hazardous inspections, and enhancing productivity across industries. Yet despite recent hardware advancements, robot learning remains far behind the progress seen in computer vision and natural language processing. While vision and language models benefit from massive web-scale datasets, robots still rely … Continue reading "\u200b"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/\" \/>\n<meta property=\"og:site_name\" content=\"4D reconstruction From Monocular Videos\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-13T03:23:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-content\/uploads\/sites\/118\/2025\/05\/Problem1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"2750\" \/>\n\t<meta property=\"og:image:height\" content=\"877\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/\",\"name\":\"\u200b - 4D reconstruction From Monocular Videos\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/wp-content\\\/uploads\\\/sites\\\/118\\\/2025\\\/05\\\/Problem1-1024x327.png\",\"datePublished\":\"2025-05-06T03:06:49+00:00\",\"dateModified\":\"2025-12-13T03:23:38+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/wp-content\\\/uploads\\\/sites\\\/118\\\/2025\\\/05\\\/Problem1.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/wp-content\\\/uploads\\\/sites\\\/118\\\/2025\\\/05\\\/Problem1.png\",\"width\":2750,\"height\":877},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u200b\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/\",\"name\":\"4D reconstruction From Monocular Videos\",\"description\":\"Students: Yu Jin Goh, Yu Hsuan Li | Advisors: Deva Ramanan (CMU), Zachary Manchester (CMU), Gengshan Yang (World Labs), John Zhang (CMU)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team13\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u200b - 4D reconstruction From Monocular Videos","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","og_locale":"en_US","og_type":"article","og_title":"\u200b - 4D reconstruction From Monocular Videos","og_description":"Overview Motivation Robots hold immense promise in transforming society\u2014supporting disaster response assisting in eldercare, performing hazardous inspections, and enhancing productivity across industries. Yet despite recent hardware advancements, robot learning remains far behind the progress seen in computer vision and natural language processing. While vision and language models benefit from massive web-scale datasets, robots still rely … Continue reading \"\u200b\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","og_site_name":"4D reconstruction From Monocular Videos","article_modified_time":"2025-12-13T03:23:38+00:00","og_image":[{"width":2750,"height":877,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-content\/uploads\/sites\/118\/2025\/05\/Problem1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","name":"\u200b - 4D reconstruction From Monocular Videos","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-content\/uploads\/sites\/118\/2025\/05\/Problem1-1024x327.png","datePublished":"2025-05-06T03:06:49+00:00","dateModified":"2025-12-13T03:23:38+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-content\/uploads\/sites\/118\/2025\/05\/Problem1.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-content\/uploads\/sites\/118\/2025\/05\/Problem1.png","width":2750,"height":877},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/"},{"@type":"ListItem","position":2,"name":"\u200b"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/","name":"4D reconstruction From Monocular Videos","description":"Students: Yu Jin Goh, Yu Hsuan Li | Advisors: Deva Ramanan (CMU), Zachary Manchester (CMU), Gengshan Yang (World Labs), John Zhang (CMU)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/pages\/8","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/users\/228"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/comments?post=8"}],"version-history":[{"count":29,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/pages\/8\/revisions"}],"predecessor-version":[{"id":212,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/pages\/8\/revisions\/212"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team13\/wp-json\/wp\/v2\/media?parent=8"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}