{"id":14,"date":"2026-05-07T01:28:33","date_gmt":"2026-05-07T01:28:33","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/?page_id=14"},"modified":"2026-05-07T17:01:59","modified_gmt":"2026-05-07T17:01:59","slug":"overview","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","title":{"rendered":"Overview"},"content":{"rendered":"\n

Motivation<\/h2>\n\n\n\n

Clay shaping is a contact-rich manipulation task where small hand and finger motions directly change the shape of the material over time. A normal video captures appearance, but it misses the underlying process: how the hands move, where contact occurs, and how the clay geometry evolves. Our goal is to build a capture system that can recover this full 4D process, supporting applications in digital preservation, skill analysis, and interactive learning.<\/p>\n\n\n\n

Problem Summary<\/h2>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Given synchronized multiview RGB video and contact sensing, we aim to recover a temporally consistent 4D representation of a person shaping clay. This includes the body pose, detailed hand pose, hand\u2013clay contact, and physics-grounded deformable clay geometry. The key difficulty is that these outputs are strongly coupled: hand motion causes contact, contact drives deformation, and deformation changes the visible geometry over time. In this project, we aim to recover body pose, hand pose, hand\u2013clay contact, and physics-grounded clay geometry from multiview RGB video.<\/p>\n\n\n\n

Challenges<\/h2>\n\n\n\n
\n
\n
\"\"
Original State<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Initial Shaping<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Complex Pose<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Heavy Occlusion<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

This problem is difficult because hands and clay frequently occlude each other, especially during fine manipulation. Clay is also soft and deformable, so its shape changes continuously rather than behaving like a rigid object. In addition, pottery involves precise and sometimes interlaced finger motions that are hard to observe from cameras alone. A practical system also needs to remain physically plausible while avoiding a capture setup that is too invasive or difficult to deploy.<\/p>\n","protected":false},"excerpt":{"rendered":"

Motivation Clay shaping is a contact-rich manipulation task where small hand and finger motions directly change the shape of the material over time. A normal video captures appearance, but it misses the underlying process: how the hands move, where contact occurs, and how the clay geometry evolves. Our goal is to build a capture system … <\/p>\n

Continue reading “Overview”<\/span><\/a><\/p>\n","protected":false},"author":307,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-14","page","type-page","status-publish","hentry"],"yoast_head":"\nOverview - Joint 4D Human and Deformable Object Capture Studio<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Overview - Joint 4D Human and Deformable Object Capture Studio\" \/>\n<meta property=\"og:description\" content=\"Motivation Clay shaping is a contact-rich manipulation task where small hand and finger motions directly change the shape of the material over time. A normal video captures appearance, but it misses the underlying process: how the hands move, where contact occurs, and how the clay geometry evolves. Our goal is to build a capture system … Continue reading "Overview"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/\" \/>\n<meta property=\"og:site_name\" content=\"Joint 4D Human and Deformable Object Capture Studio\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-07T17:01:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/hero2.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"490\" \/>\n\t<meta property=\"og:image:height\" content=\"430\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\",\"name\":\"Overview - Joint 4D Human and Deformable Object Capture Studio\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/hero2.jpg\",\"datePublished\":\"2026-05-07T01:28:33+00:00\",\"dateModified\":\"2026-05-07T17:01:59+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/hero2.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/hero2.jpg\",\"width\":490,\"height\":430},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\",\"name\":\"Joint 4D Human and Deformable Object Capture Studio\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Overview - Joint 4D Human and Deformable Object Capture Studio","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","og_locale":"en_US","og_type":"article","og_title":"Overview - Joint 4D Human and Deformable Object Capture Studio","og_description":"Motivation Clay shaping is a contact-rich manipulation task where small hand and finger motions directly change the shape of the material over time. A normal video captures appearance, but it misses the underlying process: how the hands move, where contact occurs, and how the clay geometry evolves. Our goal is to build a capture system … Continue reading \"Overview\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","og_site_name":"Joint 4D Human and Deformable Object Capture Studio","article_modified_time":"2026-05-07T17:01:59+00:00","og_image":[{"width":490,"height":430,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/hero2.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","name":"Overview - Joint 4D Human and Deformable Object Capture Studio","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/hero2.jpg","datePublished":"2026-05-07T01:28:33+00:00","dateModified":"2026-05-07T17:01:59+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/hero2.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/hero2.jpg","width":490,"height":430},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/"},{"@type":"ListItem","position":2,"name":"Overview"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","name":"Joint 4D Human and Deformable Object Capture Studio","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/14","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/users\/307"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/comments?post=14"}],"version-history":[{"count":2,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/14\/revisions"}],"predecessor-version":[{"id":49,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/14\/revisions\/49"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/media?parent=14"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}