{"id":93,"date":"2022-05-02T02:19:31","date_gmt":"2022-05-02T02:19:31","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/?page_id=93"},"modified":"2022-10-07T04:53:47","modified_gmt":"2022-10-07T04:53:47","slug":"applications","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/","title":{"rendered":"Applications"},"content":{"rendered":"\n

1. GAN Inversion<\/strong><\/h2>\n\n\n\n

For image inversion, we use 15 StyleGAN2-based models at 256×256 resolution, for a fair comparison regarding image resolution and network architectures. We first evaluate real image inversion on the validation set images from LSUN Church and CelebA-HQ datasets. We use the optimization-based inversion technique in \ud835\udc4a + latent space with LPIPS and pixel level mean square loss between the target and generated image. Given an image query, we run inversion on models ranked at 1, 10, and 15 by our Gaussian Density retrieval method. <\/p>\n\n\n\n

We evaluate the reconstruction quality between 100 inverted and target images of both categories using standard metrics like LPIPS and PSNR. We also calculate the mean distance between optimized w+ opt and the mean latent w of the model which shows the extent of overfitting to the reconstruction loss by the model. The results are shown in Table 4. We observe that top retrieved models that are similar to the image query result in better image inversion on average across all metrics. Moreover, for lower-ranked models, the distance ||w+ opt \u2212 w_avg || is significantly higher compared to the top-rank models, which has been shown to correlate negatively with image editability. Figure 8 shows some qualitative samples of image inversion using the different ranked models. Lower-ranked models yield inversions with substantially poorer quality.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n

2. Image editing and interpolation<\/strong><\/h2>\n\n\n\n

We now show that images inverted with top-ranked models can be further edited using existing GAN-based image editing techniques such as GANSpace. Figure 9 shows examples of editing on Ukiyo-e images to change the frowning face to a smiling face. For the flower category, we show example edits that change the petal colors. We can also perform latent space interpolation between inverted images of the same category and create visually compelling samples as shown in Figure 11. A rank-1 model results in the smoother interpolation of one image into another in contrast to inversion and interpolation using a random model.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n

3. Few-Shot Transfer Learning<\/strong><\/h2>\n\n\n\n

For few-shot transfer learning, we restrict our experiments to 256×256 resolution StyleGAN2 models due to limited computing resources (27 models). We begin with a small dataset of 100-136 images. Then, we rank the 27 models by the average retrieval score over all images. We select models at ranks 1, 20, and 25 as source models for transfer
learning. For finetuning the generator on the new dataset, we use vision-aided GANs, one of the leading methods in few-shot GAN training. <\/p>\n\n\n\n

Datasets and evaluation metric:<\/strong> We use three standard few-shot datasets: Obama (100 images), Grumpy Cat (100 images), and Moongate. (136 images). We use the Fr\u00e9chet Inception Distance (FID) metric for evaluation.<\/p>\n\n\n\n

Results:<\/strong> Figure 10 shows the results of transfer learning using different source models with varying retrieval rankings. We observe on average faster convergence when fine-tuning from rank-1 models compared to other lower-ranked models. This shows empirically that training from similar models results in faster convergence and thus requires less compute.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n","protected":false},"excerpt":{"rendered":"

1. GAN Inversion For image inversion, we use 15 StyleGAN2-based models at 256×256 resolution, for a fair comparison regarding image resolution and network architectures. We first evaluate real image inversion on the validation set images from LSUN Church and CelebA-HQ datasets. We use the optimization-based inversion technique in \ud835\udc4a + latent space with LPIPS and … <\/p>\n

Continue reading “Applications”<\/span><\/a><\/p>\n","protected":false},"author":135,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-93","page","type-page","status-publish","hentry"],"yoast_head":"\nApplications - One Million Generative Models<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Applications - One Million Generative Models\" \/>\n<meta property=\"og:description\" content=\"1. GAN Inversion For image inversion, we use 15 StyleGAN2-based models at 256×256 resolution, for a fair comparison regarding image resolution and network architectures. We first evaluate real image inversion on the validation set images from LSUN Church and CelebA-HQ datasets. We use the optimization-based inversion technique in \ud835\udc4a + latent space with LPIPS and … Continue reading "Applications"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/\" \/>\n<meta property=\"og:site_name\" content=\"One Million Generative Models\" \/>\n<meta property=\"article:modified_time\" content=\"2022-10-07T04:53:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/09\/Screen-Shot-2022-09-06-at-7.25.00-PM-1024x240.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/\",\"name\":\"Applications - One Million Generative Models\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/09\\\/Screen-Shot-2022-09-06-at-7.25.00-PM-1024x240.png\",\"datePublished\":\"2022-05-02T02:19:31+00:00\",\"dateModified\":\"2022-10-07T04:53:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/09\\\/Screen-Shot-2022-09-06-at-7.25.00-PM.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/09\\\/Screen-Shot-2022-09-06-at-7.25.00-PM.png\",\"width\":2266,\"height\":532},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/applications\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Applications\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\",\"name\":\"One Million Generative Models\",\"description\":\"Students: Daohan "Fred" Lu (CMU), Rohan Agarwal (CMU) | Collaborators: Nupur Kumari (CMU), Sheng-Yu Wang (CMU) | Advisors: Jun-Yan Zhu (CMU), David Bau (Northeastern University)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Applications - One Million Generative Models","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/","og_locale":"en_US","og_type":"article","og_title":"Applications - One Million Generative Models","og_description":"1. GAN Inversion For image inversion, we use 15 StyleGAN2-based models at 256×256 resolution, for a fair comparison regarding image resolution and network architectures. We first evaluate real image inversion on the validation set images from LSUN Church and CelebA-HQ datasets. We use the optimization-based inversion technique in \ud835\udc4a + latent space with LPIPS and … Continue reading \"Applications\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/","og_site_name":"One Million Generative Models","article_modified_time":"2022-10-07T04:53:47+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/09\/Screen-Shot-2022-09-06-at-7.25.00-PM-1024x240.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/","name":"Applications - One Million Generative Models","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/09\/Screen-Shot-2022-09-06-at-7.25.00-PM-1024x240.png","datePublished":"2022-05-02T02:19:31+00:00","dateModified":"2022-10-07T04:53:47+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/09\/Screen-Shot-2022-09-06-at-7.25.00-PM.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/09\/Screen-Shot-2022-09-06-at-7.25.00-PM.png","width":2266,"height":532},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/applications\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/"},{"@type":"ListItem","position":2,"name":"Applications"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","name":"One Million Generative Models","description":"Students: Daohan "Fred" Lu (CMU), Rohan Agarwal (CMU) | Collaborators: Nupur Kumari (CMU), Sheng-Yu Wang (CMU) | Advisors: Jun-Yan Zhu (CMU), David Bau (Northeastern University)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/93","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/users\/135"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/comments?post=93"}],"version-history":[{"count":7,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/93\/revisions"}],"predecessor-version":[{"id":247,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/93\/revisions\/247"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/media?parent=93"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}