{"id":9,"date":"2026-05-06T21:20:04","date_gmt":"2026-05-06T21:20:04","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/?page_id=9"},"modified":"2026-05-08T02:33:45","modified_gmt":"2026-05-08T02:33:45","slug":"method","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/","title":{"rendered":"Method"},"content":{"rendered":"\n

Method<\/strong><\/h2>\n\n\n\n

Non-rigging<\/strong><\/h3>\n\n\n\n

Our method aims to learn an optimal canonical mesh M*<\/strong> that could correctly deform at each timestep. We achieve this by optimizing the vertex positions of the canonical mesh to minimize both geometric loss and photometric loss. We use the frozen ActionMesh’s deformation decoder to predict per-timestep deformation. For geometric loss, we first generate a sequence of temporally consistent independent meshes, then compute the average chamfer distance between the deformed canonical mesh and the independent mesh at each timestep. For photometric loss, we compute the average L1 loss between the rasterized images and the input video.

For independent mesh generation, we implement a stochastic latent propagation<\/strong> strategy inspired by SDEdit <\/strong>[4], where the latent from timestep t is perturbed with Gaussian noise and then denoised to form the initialization for timestep t+1 & t-1.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Rigging<\/strong><\/h3>\n\n\n\n

We propose a pipeline that adapt the skeleton generation pipeline from puppeteer[5], which uses auto-regressive transformer to generation skeleton taking object mesh as input. Then we’ll use a user editing module which takes in user’s operation (ADD, MOVE, REMOVE), generate modified skeleton with refinement transformer.<\/p>\n\n\n\n

For the first semester, we tried deterministic skeleton editing method, and we will do non-deterministic pipeline or use the editing to guide the skeleton generation.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

References<\/strong><\/h3>\n\n\n\n

[4] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon: SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. ICLR 2022<\/p>\n\n\n\n

[5] Song, Chaoyue, Xiu Li, Fan Yang, Zhongcong Xu, Jiacheng Wei, Fayao Liu, Jiashi Feng, Guosheng Lin, and Jianfeng Zhang. “Puppeteer: Rig and animate your 3d models.” 2025 arXiv:2508.10898 (2025)<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Method Non-rigging Our method aims to learn an optimal canonical mesh M* that could correctly deform at each timestep. We achieve this by optimizing the vertex positions of the canonical mesh to minimize both geometric loss and photometric loss. We use the frozen ActionMesh’s deformation decoder to predict per-timestep deformation. For geometric loss, we first […]<\/p>\n","protected":false},"author":270,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-9","page","type-page","status-publish","hentry"],"yoast_head":"\nMethod - 3D Asset Generation: Generating Action Primitives<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Method - 3D Asset Generation: Generating Action Primitives\" \/>\n<meta property=\"og:description\" content=\"Method Non-rigging Our method aims to learn an optimal canonical mesh M* that could correctly deform at each timestep. We achieve this by optimizing the vertex positions of the canonical mesh to minimize both geometric loss and photometric loss. We use the frozen ActionMesh’s deformation decoder to predict per-timestep deformation. For geometric loss, we first […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/\" \/>\n<meta property=\"og:site_name\" content=\"3D Asset Generation: Generating Action Primitives\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-08T02:33:45+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-content\/uploads\/sites\/145\/2026\/05\/img4.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1095\" \/>\n\t<meta property=\"og:image:height\" content=\"291\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/\",\"name\":\"Method - 3D Asset Generation: Generating Action Primitives\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/wp-content\\\/uploads\\\/sites\\\/145\\\/2026\\\/05\\\/img4-1024x272.png\",\"datePublished\":\"2026-05-06T21:20:04+00:00\",\"dateModified\":\"2026-05-08T02:33:45+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/wp-content\\\/uploads\\\/sites\\\/145\\\/2026\\\/05\\\/img4.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/wp-content\\\/uploads\\\/sites\\\/145\\\/2026\\\/05\\\/img4.png\",\"width\":1095,\"height\":291},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/method\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Method\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/\",\"name\":\"3D Asset Generation: Generating Action Primitives\",\"description\":\"Carnegie Mellon University\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf3\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Method - 3D Asset Generation: Generating Action Primitives","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/","og_locale":"en_US","og_type":"article","og_title":"Method - 3D Asset Generation: Generating Action Primitives","og_description":"Method Non-rigging Our method aims to learn an optimal canonical mesh M* that could correctly deform at each timestep. We achieve this by optimizing the vertex positions of the canonical mesh to minimize both geometric loss and photometric loss. We use the frozen ActionMesh’s deformation decoder to predict per-timestep deformation. For geometric loss, we first […]","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/","og_site_name":"3D Asset Generation: Generating Action Primitives","article_modified_time":"2026-05-08T02:33:45+00:00","og_image":[{"width":1095,"height":291,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-content\/uploads\/sites\/145\/2026\/05\/img4.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/","name":"Method - 3D Asset Generation: Generating Action Primitives","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-content\/uploads\/sites\/145\/2026\/05\/img4-1024x272.png","datePublished":"2026-05-06T21:20:04+00:00","dateModified":"2026-05-08T02:33:45+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-content\/uploads\/sites\/145\/2026\/05\/img4.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-content\/uploads\/sites\/145\/2026\/05\/img4.png","width":1095,"height":291},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/method\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/"},{"@type":"ListItem","position":2,"name":"Method"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/","name":"3D Asset Generation: Generating Action Primitives","description":"Carnegie Mellon University","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/pages\/9","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/users\/270"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/comments?post=9"}],"version-history":[{"count":14,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/pages\/9\/revisions"}],"predecessor-version":[{"id":167,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/pages\/9\/revisions\/167"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf3\/wp-json\/wp\/v2\/media?parent=9"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}