{"id":7,"date":"2022-05-01T18:05:30","date_gmt":"2022-05-01T18:05:30","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/?page_id=7"},"modified":"2022-12-21T04:38:48","modified_gmt":"2022-12-21T04:38:48","slug":"introduction","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","title":{"rendered":"Introduction"},"content":{"rendered":"\n

ArXiv<\/a><\/mark> <\/mark>|<\/mark> <\/mark>Github<\/a><\/mark> <\/mark>|<\/mark> <\/mark>Modelverse<\/a><\/mark><\/strong> | Youtube<\/a><\/strong><\/p>\n\n\n\n

\"\"
We develop a content-based search engine for Modelverse, a model-sharing platform that contains a diverse set of deep generative models, such as animals, landscapes, portraits, and art pieces. From left to right, our search algorithm enables queries (1st row) with four different modalities \u2013 text, images, sketches, and existing models. The 2nd and 3rd rows show the two top-ranked models. The color of each model icon implies the model type. Our method finds relevant models with similar semantic concepts in all modalities.<\/figcaption><\/figure>\n\n\n\n

The growing proliferation of pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, find the models that best match the query. Because each generative model produces a distribution of images, we formulate the search problem as an optimization to maximize the probability of generating a query match given a model. We develop approximations to make this problem tractable when the query is an image, a sketch, a text description, another generative model, or a combination of the above. We benchmark our method in both accuracy and speed over a set of generative models. We demonstrate that our model search retrieves suitable models for image editing and reconstruction, few-shot transfer learning, and latent space interpolation. Finally, we deploy our search algorithm to our online generative model-sharing platform at https:\/\/modelverse.cs.cmu.edu\/<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

ArXiv | Github | Modelverse | Youtube The growing proliferation of pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, find the models … <\/p>\n

Continue reading “Introduction”<\/span><\/a><\/p>\n","protected":false},"author":135,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-7","page","type-page","status-publish","hentry"],"yoast_head":"\nIntroduction - One Million Generative Models<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction - One Million Generative Models\" \/>\n<meta property=\"og:description\" content=\"ArXiv | Github | Modelverse | Youtube The growing proliferation of pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, find the models … Continue reading "Introduction"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/\" \/>\n<meta property=\"og:site_name\" content=\"One Million Generative Models\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-21T04:38:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/12\/teaser-1-1024x634.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\",\"name\":\"Introduction - One Million Generative Models\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/12\\\/teaser-1-1024x634.jpg\",\"datePublished\":\"2022-05-01T18:05:30+00:00\",\"dateModified\":\"2022-12-21T04:38:48+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/12\\\/teaser-1.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/wp-content\\\/uploads\\\/sites\\\/63\\\/2022\\\/12\\\/teaser-1.jpg\",\"width\":1450,\"height\":898},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/\",\"name\":\"One Million Generative Models\",\"description\":\"Students: Daohan "Fred" Lu (CMU), Rohan Agarwal (CMU) | Collaborators: Nupur Kumari (CMU), Sheng-Yu Wang (CMU) | Advisors: Jun-Yan Zhu (CMU), David Bau (Northeastern University)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team8\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction - One Million Generative Models","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","og_locale":"en_US","og_type":"article","og_title":"Introduction - One Million Generative Models","og_description":"ArXiv | Github | Modelverse | Youtube The growing proliferation of pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, find the models … Continue reading \"Introduction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","og_site_name":"One Million Generative Models","article_modified_time":"2022-12-21T04:38:48+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/12\/teaser-1-1024x634.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","name":"Introduction - One Million Generative Models","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/12\/teaser-1-1024x634.jpg","datePublished":"2022-05-01T18:05:30+00:00","dateModified":"2022-12-21T04:38:48+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/12\/teaser-1.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-content\/uploads\/sites\/63\/2022\/12\/teaser-1.jpg","width":1450,"height":898},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/"},{"@type":"ListItem","position":2,"name":"Introduction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/","name":"One Million Generative Models","description":"Students: Daohan "Fred" Lu (CMU), Rohan Agarwal (CMU) | Collaborators: Nupur Kumari (CMU), Sheng-Yu Wang (CMU) | Advisors: Jun-Yan Zhu (CMU), David Bau (Northeastern University)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/7","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/users\/135"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":29,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/7\/revisions"}],"predecessor-version":[{"id":301,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/pages\/7\/revisions\/301"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team8\/wp-json\/wp\/v2\/media?parent=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}