{"id":19,"date":"2025-05-09T20:21:31","date_gmt":"2025-05-09T20:21:31","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/?page_id=19"},"modified":"2025-12-12T22:10:22","modified_gmt":"2025-12-12T22:10:22","slug":"methods","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/","title":{"rendered":"Method"},"content":{"rendered":"\n

Cascaded Autoencoder for Smooth HOI Latents<\/strong><\/h4>\n\n\n\n

As illustrated in\u00a0Figure 1<\/strong>, we introduce a cascaded autoencoder that learns continuous latent representations for hand\u2013object interaction motion by separately encoding object motion and articulated hand trajectories while preserving their interaction structure. This design enables the latent space to capture fine-grained temporal dynamics and avoids the quantization artifacts commonly introduced by discrete VQ-based models. To evaluate motion smoothness, we perform a jerk analysis (Figure 2<\/strong>), where jerk is defined as the third derivative of position with respect to time. As shown across multiple temporal window lengths, our autoencoder consistently yields lower jerk values than VQ-based baselines, indicating smoother, more natural, and temporally stable HOI reconstructions.<\/p>\n\n\n\n

\"\"
Figure 1<\/strong>. Our cascaded autoencoder separately encodes object motion and articulated hand trajectories into continuous latent spaces, which are jointly decoded to reconstruct coherent hand\u2013object interactio<\/figcaption><\/figure>\n\n\n\n
\"\"
Figure 2<\/strong>. We compare jerk profiles (third derivative of position) between our autoencoder and VQ-based models across different temporal window lengths, showing consistently lower jerk values for our approach, indicating smoother and more temporally stable reconstructions.<\/figcaption><\/figure>\n\n\n\n

Latent Diffusion for HOI Motion Generation<\/strong><\/h4>\n\n\n\n

We introduce a latent diffusion framework for hand\u2013object interaction motion generation that combines the strengths of diffusion and autoregressive modeling. As shown in Figure 3<\/strong>, an autoregressive transformer models the compositional structure of HOI motion in latent space, while a diffusion module refines per-token distributions to produce smooth and expressive motion trajectories. This design preserves the flexibility of autoregressive generation for arbitrary-length sequences while enabling continuous, high-fidelity motion synthesis under multimodal conditioning such as language.<\/p>\n\n\n

\n
\"\"
Figure 3. <\/strong>An autoregressive transformer predicts latent tokens conditioned on multimodal inputs, while a diffusion module predicts masked token to produce smooth, continuous HOI motion representations.<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
Table 1. <\/strong>Our latent diffusion approach combines arbitrary-length generation from autoregressive models with the smoothness and continuous representations of diffusion, enabling high-fidelity HOI motion synthesis.<\/figcaption><\/figure>\n<\/div>","protected":false},"excerpt":{"rendered":"

Cascaded Autoencoder for Smooth HOI Latents As illustrated in\u00a0Figure 1, we introduce a cascaded autoencoder that learns continuous latent representations for hand\u2013object interaction motion by separately encoding object motion and articulated hand trajectories while preserving their interaction structure. This design enables the latent space to capture fine-grained temporal dynamics and avoids the quantization artifacts commonly … <\/p>\n

Continue reading “Method”<\/span><\/a><\/p>\n","protected":false},"author":237,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-19","page","type-page","status-publish","hentry"],"yoast_head":"\nMethod - Text-Driven Motion Generation for Hand-Object Interaction<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Method - Text-Driven Motion Generation for Hand-Object Interaction\" \/>\n<meta property=\"og:description\" content=\"Cascaded Autoencoder for Smooth HOI Latents As illustrated in\u00a0Figure 1, we introduce a cascaded autoencoder that learns continuous latent representations for hand\u2013object interaction motion by separately encoding object motion and articulated hand trajectories while preserving their interaction structure. This design enables the latent space to capture fine-grained temporal dynamics and avoids the quantization artifacts commonly … Continue reading "Method"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/\" \/>\n<meta property=\"og:site_name\" content=\"Text-Driven Motion Generation for Hand-Object Interaction\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-12T22:10:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-content\/uploads\/sites\/123\/2025\/12\/Screenshot-2025-12-12-at-4.57.08-PM-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"726\" \/>\n\t<meta property=\"og:image:height\" content=\"590\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/\",\"name\":\"Method - Text-Driven Motion Generation for Hand-Object Interaction\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/wp-content\\\/uploads\\\/sites\\\/123\\\/2025\\\/12\\\/Screenshot-2025-12-12-at-4.57.08-PM-1.png\",\"datePublished\":\"2025-05-09T20:21:31+00:00\",\"dateModified\":\"2025-12-12T22:10:22+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/wp-content\\\/uploads\\\/sites\\\/123\\\/2025\\\/12\\\/Screenshot-2025-12-12-at-4.57.08-PM-1.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/wp-content\\\/uploads\\\/sites\\\/123\\\/2025\\\/12\\\/Screenshot-2025-12-12-at-4.57.08-PM-1.png\",\"width\":726,\"height\":590},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/methods\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Method\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/\",\"name\":\"Text-Driven Motion Generation for Hand-Object Interaction\",\"description\":\"CMU MSCV Capstone Project | Ethan Lai | Advisors: L\u00e1szl\u00f3 A. Jeni and Ananya Bal\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team10\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Method - Text-Driven Motion Generation for Hand-Object Interaction","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/","og_locale":"en_US","og_type":"article","og_title":"Method - Text-Driven Motion Generation for Hand-Object Interaction","og_description":"Cascaded Autoencoder for Smooth HOI Latents As illustrated in\u00a0Figure 1, we introduce a cascaded autoencoder that learns continuous latent representations for hand\u2013object interaction motion by separately encoding object motion and articulated hand trajectories while preserving their interaction structure. This design enables the latent space to capture fine-grained temporal dynamics and avoids the quantization artifacts commonly … Continue reading \"Method\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/","og_site_name":"Text-Driven Motion Generation for Hand-Object Interaction","article_modified_time":"2025-12-12T22:10:22+00:00","og_image":[{"width":726,"height":590,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-content\/uploads\/sites\/123\/2025\/12\/Screenshot-2025-12-12-at-4.57.08-PM-1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/","name":"Method - Text-Driven Motion Generation for Hand-Object Interaction","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-content\/uploads\/sites\/123\/2025\/12\/Screenshot-2025-12-12-at-4.57.08-PM-1.png","datePublished":"2025-05-09T20:21:31+00:00","dateModified":"2025-12-12T22:10:22+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-content\/uploads\/sites\/123\/2025\/12\/Screenshot-2025-12-12-at-4.57.08-PM-1.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-content\/uploads\/sites\/123\/2025\/12\/Screenshot-2025-12-12-at-4.57.08-PM-1.png","width":726,"height":590},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/methods\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/"},{"@type":"ListItem","position":2,"name":"Method"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/","name":"Text-Driven Motion Generation for Hand-Object Interaction","description":"CMU MSCV Capstone Project | Ethan Lai | Advisors: L\u00e1szl\u00f3 A. Jeni and Ananya Bal","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/pages\/19","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/users\/237"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/comments?post=19"}],"version-history":[{"count":8,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/pages\/19\/revisions"}],"predecessor-version":[{"id":77,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/pages\/19\/revisions\/77"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team10\/wp-json\/wp\/v2\/media?parent=19"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}