{"id":9,"date":"2025-05-09T21:20:26","date_gmt":"2025-05-09T21:20:26","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/?page_id=9"},"modified":"2025-12-10T04:45:44","modified_gmt":"2025-12-10T04:45:44","slug":"introduction","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/","title":{"rendered":"Introduction"},"content":{"rendered":"\n<p>We introduce a conditional video distillation method for real-time co-speech video generation that leverages human pose conditioning for input-aware sparse attention and distillation loss. Our student model achieves 25.3 FPS, a 13.1x speedup over its teacher model, while preserving visual quality. Our method significantly improves motion coherence and lip synchronization over a leading few-step causal student model while reducing common visual degradation in the speaker&#8217;s face and hands (see yellow box). Example frames are from our curated YouTube Talking Video dataset \u00a9 TED Conferences, LLC.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"379\" height=\"242\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png\" alt=\"\" class=\"wp-image-62\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png 379w, https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11-300x192.png 300w\" sizes=\"auto, (max-width: 379px) 100vw, 379px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Background<\/h2>\n\n\n\n<p>Video generation is a fundamental task in computer vision that aims to synthesize realistic and temporally coherent video sequences from various inputs such as text, audio, poses, or static images. It holds significant potential in numerous applications, including content creation, virtual reality, digital humans, and simulation. Despite recent progress, generating <strong>long, high-quality, and controllable<\/strong> videos remains highly challenging. Existing models often struggle with temporal consistency, motion realism, and computational efficiency, particularly when conditioned on multi-modal inputs like speech or gestures. Moreover, most approaches focus on <strong>offline generation<\/strong>, limiting their usability in <strong>real-time or interactive<\/strong> scenarios.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"648\" height=\"108\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/05\/Picture1.gif\" alt=\"\" class=\"wp-image-29\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"122\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/05\/Picture3.gif\" alt=\"\" class=\"wp-image-32\" style=\"width:645px;height:auto\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>We introduce a conditional video distillation method for real-time co-speech video generation that leverages human pose conditioning for input-aware sparse attention and distillation loss. Our student model achieves 25.3 FPS, a 13.1x speedup over its teacher model, while preserving visual quality. Our method significantly improves motion coherence and lip synchronization over a leading few-step causal &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Introduction&#8221;<\/span><\/a><\/p>\n","protected":false},"author":260,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-9","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Introduction - 2025 Team 16-1 Project<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction - 2025 Team 16-1 Project\" \/>\n<meta property=\"og:description\" content=\"We introduce a conditional video distillation method for real-time co-speech video generation that leverages human pose conditioning for input-aware sparse attention and distillation loss. Our student model achieves 25.3 FPS, a 13.1x speedup over its teacher model, while preserving visual quality. Our method significantly improves motion coherence and lip synchronization over a leading few-step causal &hellip; Continue reading &quot;Introduction&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/\" \/>\n<meta property=\"og:site_name\" content=\"2025 Team 16-1 Project\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-10T04:45:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png\" \/>\n\t<meta property=\"og:image:width\" content=\"379\" \/>\n\t<meta property=\"og:image:height\" content=\"242\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/\",\"name\":\"Introduction - 2025 Team 16-1 Project\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/wp-content\\\/uploads\\\/sites\\\/138\\\/2025\\\/12\\\/11.png\",\"datePublished\":\"2025-05-09T21:20:26+00:00\",\"dateModified\":\"2025-12-10T04:45:44+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/wp-content\\\/uploads\\\/sites\\\/138\\\/2025\\\/12\\\/11.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/wp-content\\\/uploads\\\/sites\\\/138\\\/2025\\\/12\\\/11.png\",\"width\":379,\"height\":242},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/introduction\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/\",\"name\":\"2025 Team 16-1 Project\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team16-1\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction - 2025 Team 16-1 Project","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/","og_locale":"en_US","og_type":"article","og_title":"Introduction - 2025 Team 16-1 Project","og_description":"We introduce a conditional video distillation method for real-time co-speech video generation that leverages human pose conditioning for input-aware sparse attention and distillation loss. Our student model achieves 25.3 FPS, a 13.1x speedup over its teacher model, while preserving visual quality. Our method significantly improves motion coherence and lip synchronization over a leading few-step causal &hellip; Continue reading \"Introduction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/","og_site_name":"2025 Team 16-1 Project","article_modified_time":"2025-12-10T04:45:44+00:00","og_image":[{"width":379,"height":242,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/","name":"Introduction - 2025 Team 16-1 Project","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png","datePublished":"2025-05-09T21:20:26+00:00","dateModified":"2025-12-10T04:45:44+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-content\/uploads\/sites\/138\/2025\/12\/11.png","width":379,"height":242},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/introduction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/"},{"@type":"ListItem","position":2,"name":"Introduction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/","name":"2025 Team 16-1 Project","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/pages\/9","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/users\/260"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/comments?post=9"}],"version-history":[{"count":7,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/pages\/9\/revisions"}],"predecessor-version":[{"id":63,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/pages\/9\/revisions\/63"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-1\/wp-json\/wp\/v2\/media?parent=9"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}