{"id":18,"date":"2026-05-07T01:30:11","date_gmt":"2026-05-07T01:30:11","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/?page_id=18"},"modified":"2026-05-07T17:10:54","modified_gmt":"2026-05-07T17:10:54","slug":"method","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/","title":{"rendered":"Method"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Our proposed pipeline combines the three main directions from the related work: multiview human capture, contact-aware hand-object reasoning, and physics-grounded deformable object modeling.<\/p>\n\n\n\n

First, we use Contact4D<\/strong> as the main reference for recovering a consistent parametric human model from multiview video. In our pipeline, this corresponds to estimating the body pose and detailed hand pose across time. Its multiview triangulation and refinement ideas are useful because clay shaping involves frequent hand occlusions, so single-view pose estimates will not be reliable enough.<\/p>\n\n\n\n

Next, we use DyTact<\/strong> as inspiration for estimating hand-object contact from visual evidence. In our setting, this means identifying where the hands touch the clay and using that contact information to guide reconstruction. This contact step is important because the clay deformation is not random; it is caused by local hand pressure and motion.<\/p>\n\n\n\n

Finally, we use GIC<\/strong> as the foundation for reconstructing the clay as a deformable, physics-grounded object. Instead of treating the clay as only a surface mesh or visual reconstruction, we want a representation that can evolve over time in a physically meaningful way. GIC\u2019s use of dynamic 3D Gaussians, material particles, and simulation gives us a direction for modeling clay-like deformation.<\/p>\n\n\n\n

Together, these components form a joint 4D capture pipeline: Contact4D provides the human pose, DyTact provides contact reasoning, and GIC provides deformable clay reconstruction<\/strong>. The final output is a time-varying representation containing body pose, hand pose, hand\u2013clay contact, and physics-grounded clay geometry.<\/p>\n\n\n\n

Capture Setup<\/h2>\n\n\n\n
\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n

Our capture setup uses a half-dome of 17 synchronized GoPro cameras for external multiview video, along with Project Aria glasses for egocentric views. Camera calibration is performed with COLMAP using stationary views and additional mobile sweep images to improve coverage. This setup is designed to capture both the global human motion and the fine local hand\u2013clay interaction needed for reconstruction.<\/p>\n","protected":false},"excerpt":{"rendered":"

Our proposed pipeline combines the three main directions from the related work: multiview human capture, contact-aware hand-object reasoning, and physics-grounded deformable object modeling. First, we use Contact4D as the main reference for recovering a consistent parametric human model from multiview video. In our pipeline, this corresponds to estimating the body pose and detailed hand pose … <\/p>\n

Continue reading “Method”<\/span><\/a><\/p>\n","protected":false},"author":307,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-18","page","type-page","status-publish","hentry"],"yoast_head":"\nMethod - Joint 4D Human and Deformable Object Capture Studio<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Method - Joint 4D Human and Deformable Object Capture Studio\" \/>\n<meta property=\"og:description\" content=\"Our proposed pipeline combines the three main directions from the related work: multiview human capture, contact-aware hand-object reasoning, and physics-grounded deformable object modeling. First, we use Contact4D as the main reference for recovering a consistent parametric human model from multiview video. In our pipeline, this corresponds to estimating the body pose and detailed hand pose … Continue reading "Method"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/\" \/>\n<meta property=\"og:site_name\" content=\"Joint 4D Human and Deformable Object Capture Studio\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-07T17:10:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/pipeline.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1085\" \/>\n\t<meta property=\"og:image:height\" content=\"440\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/\",\"name\":\"Method - Joint 4D Human and Deformable Object Capture Studio\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/pipeline-1024x415.jpg\",\"datePublished\":\"2026-05-07T01:30:11+00:00\",\"dateModified\":\"2026-05-07T17:10:54+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/pipeline.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/wp-content\\\/uploads\\\/sites\\\/161\\\/2026\\\/05\\\/pipeline.jpg\",\"width\":1085,\"height\":440},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/method\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Method\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/\",\"name\":\"Joint 4D Human and Deformable Object Capture Studio\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf19\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Method - Joint 4D Human and Deformable Object Capture Studio","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/","og_locale":"en_US","og_type":"article","og_title":"Method - Joint 4D Human and Deformable Object Capture Studio","og_description":"Our proposed pipeline combines the three main directions from the related work: multiview human capture, contact-aware hand-object reasoning, and physics-grounded deformable object modeling. First, we use Contact4D as the main reference for recovering a consistent parametric human model from multiview video. In our pipeline, this corresponds to estimating the body pose and detailed hand pose … Continue reading \"Method\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/","og_site_name":"Joint 4D Human and Deformable Object Capture Studio","article_modified_time":"2026-05-07T17:10:54+00:00","og_image":[{"width":1085,"height":440,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/pipeline.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/","name":"Method - Joint 4D Human and Deformable Object Capture Studio","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/pipeline-1024x415.jpg","datePublished":"2026-05-07T01:30:11+00:00","dateModified":"2026-05-07T17:10:54+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/pipeline.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-content\/uploads\/sites\/161\/2026\/05\/pipeline.jpg","width":1085,"height":440},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/method\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/"},{"@type":"ListItem","position":2,"name":"Method"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/","name":"Joint 4D Human and Deformable Object Capture Studio","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/18","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/users\/307"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/comments?post=18"}],"version-history":[{"count":3,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/18\/revisions"}],"predecessor-version":[{"id":54,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/pages\/18\/revisions\/54"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf19\/wp-json\/wp\/v2\/media?parent=18"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}