{"id":35,"date":"2023-05-09T03:19:21","date_gmt":"2023-05-09T03:19:21","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/?page_id=35"},"modified":"2023-12-18T19:22:55","modified_gmt":"2023-12-18T19:22:55","slug":"results","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/","title":{"rendered":"Results"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\"><mark class=\"has-inline-color has-vivid-cyan-blue-color\">Qualitative Results<\/mark><\/h1>\n\n\n\n<figure class=\"wp-block-video aligncenter\"><video height=\"1998\" style=\"aspect-ratio: 1600 \/ 1998;\" width=\"1600\" autoplay controls loop src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/extra_all.mp4\"><\/video><figcaption class=\"wp-element-caption\"><strong>Figure 7: <\/strong>Shows qualitative results of cinemagraphs generated by our method from the text prompts (the corresponding text prompts are below the animations in each example)<\/figcaption><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\"><mark class=\"has-inline-color has-vivid-cyan-blue-color\">Qunatitative Results<\/mark><\/h1>\n\n\n\n<p>We compare our method of text-to-cinemagraph, for generating cinemagraphs from text against 2 categories of baselines:<\/p>\n\n\n\n<p>(i) image-to-cinemagraph: where we first generate image using Stable-Diffusion, and then generate cinemagraph from this image using prior works (Animating Landscape, Holynski et al. 2021, SLR-SFS)<\/p>\n\n\n\n<p>(ii) text-to-cinemagraph: where we directly generate a video from text using prior methods (Text2Video-Zero, CogVideo, VideoCrafter) and then make them looping as a post-processing step<\/p>\n\n\n\n<p>To evaluate the quality of our generated results, we perform human evaluation, where novice users on Amazon Mechanical Turk were given a text prompt asked to select which cinemagraphs between the one generated by our method and the baseline looks better. Each comparison with a baseline has recieved more than 800 annotations.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"484\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1024x484.png\" alt=\"\" class=\"wp-image-143\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1024x484.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-300x142.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-768x363.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1536x725.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image.png 1622w\" sizes=\"auto, (max-width: 706px) 89vw, (max-width: 767px) 82vw, 740px\" \/><\/figure>\n\n\n\n<p>We show a plot of what percentage, of users prefer our cinemagraphs compared to the baseline. From the plot above, we find that compared to cinemagraphs generated by both category of baselines, users prefer ones generated by our method by a large margin.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Qualitative Results Qunatitative Results We compare our method of text-to-cinemagraph, for generating cinemagraphs from text against 2 categories of baselines: (i) image-to-cinemagraph: where we first generate image using Stable-Diffusion, and then generate cinemagraph from this image using prior works (Animating Landscape, Holynski et al. 2021, SLR-SFS) (ii) text-to-cinemagraph: where we directly generate a video from &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Results&#8221;<\/span><\/a><\/p>\n","protected":false},"author":177,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-35","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Results - Adapting Stable Diffusion Models for Video Synthesis<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Results - Adapting Stable Diffusion Models for Video Synthesis\" \/>\n<meta property=\"og:description\" content=\"Qualitative Results Qunatitative Results We compare our method of text-to-cinemagraph, for generating cinemagraphs from text against 2 categories of baselines: (i) image-to-cinemagraph: where we first generate image using Stable-Diffusion, and then generate cinemagraph from this image using prior works (Animating Landscape, Holynski et al. 2021, SLR-SFS) (ii) text-to-cinemagraph: where we directly generate a video from &hellip; Continue reading &quot;Results&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/\" \/>\n<meta property=\"og:site_name\" content=\"Adapting Stable Diffusion Models for Video Synthesis\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-18T19:22:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1024x484.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/\",\"name\":\"Results - Adapting Stable Diffusion Models for Video Synthesis\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/wp-content\\\/uploads\\\/sites\\\/90\\\/2023\\\/12\\\/image-1024x484.png\",\"datePublished\":\"2023-05-09T03:19:21+00:00\",\"dateModified\":\"2023-12-18T19:22:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/wp-content\\\/uploads\\\/sites\\\/90\\\/2023\\\/12\\\/image.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/wp-content\\\/uploads\\\/sites\\\/90\\\/2023\\\/12\\\/image.png\",\"width\":1622,\"height\":766},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/results\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Results\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/\",\"name\":\"Adapting Stable Diffusion Models for Video Synthesis\",\"description\":\"Students: Aniruddha Mahapatra | Advisors: Jun-Yan Zhu (CMU), Aliaksandr Sairohin (Snap), Hsin-Ying Lee (Snap), Sergey Tulyakov (Snap) | Sponsor: CMU\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team13\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Results - Adapting Stable Diffusion Models for Video Synthesis","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/","og_locale":"en_US","og_type":"article","og_title":"Results - Adapting Stable Diffusion Models for Video Synthesis","og_description":"Qualitative Results Qunatitative Results We compare our method of text-to-cinemagraph, for generating cinemagraphs from text against 2 categories of baselines: (i) image-to-cinemagraph: where we first generate image using Stable-Diffusion, and then generate cinemagraph from this image using prior works (Animating Landscape, Holynski et al. 2021, SLR-SFS) (ii) text-to-cinemagraph: where we directly generate a video from &hellip; Continue reading \"Results\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/","og_site_name":"Adapting Stable Diffusion Models for Video Synthesis","article_modified_time":"2023-12-18T19:22:55+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1024x484.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/","name":"Results - Adapting Stable Diffusion Models for Video Synthesis","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image-1024x484.png","datePublished":"2023-05-09T03:19:21+00:00","dateModified":"2023-12-18T19:22:55+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-content\/uploads\/sites\/90\/2023\/12\/image.png","width":1622,"height":766},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/results\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/"},{"@type":"ListItem","position":2,"name":"Results"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/","name":"Adapting Stable Diffusion Models for Video Synthesis","description":"Students: Aniruddha Mahapatra | Advisors: Jun-Yan Zhu (CMU), Aliaksandr Sairohin (Snap), Hsin-Ying Lee (Snap), Sergey Tulyakov (Snap) | Sponsor: CMU","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/pages\/35","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/users\/177"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/comments?post=35"}],"version-history":[{"count":7,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/pages\/35\/revisions"}],"predecessor-version":[{"id":144,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/pages\/35\/revisions\/144"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team13\/wp-json\/wp\/v2\/media?parent=35"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}