{"id":2,"date":"2022-12-14T20:16:00","date_gmt":"2022-12-14T20:16:00","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/?page_id=2"},"modified":"2022-12-20T03:37:47","modified_gmt":"2022-12-20T03:37:47","slug":"sample-page","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","title":{"rendered":"Overview"},"content":{"rendered":"\n

Motivation<\/h2>\n\n\n\n

We are experiencing an exciting time of generative models. With the appearance and development of diffusion models [1], they are gradually replacing the prevalent generative adversarial network (GAN). Though astonishing and high-quality images are generated, there still leave many serious questions that need to be solved. Among them, fairness<\/em><\/strong> is a very crucial and nonnegligible problem. Let’s see an example.<\/p>\n\n\n\n

\"\"
Figure 1. Sampled images when giving the text “an intellectual scientist” to stable diffusion model.<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

From Figure 1, we can observe that when we ask the stable diffusion model to give me “an intellectual scientist”<\/em>, all the generated images are men wearing eyeglasses. Generally speaking, we want the generated images to be fair across the sensitive attributes, such as gender, race etc. Besides, we also remove some stereotype such as all<\/em> scientists<\/em> wear<\/em> eyeglasses<\/em>. Therefore, based on current urgent fairness problems in diffusion models, we want to solve it in this project.<\/p>\n\n\n\n

Problem Statement<\/h2>\n\n\n\n

Because of the imbalance of training dataset (LAION-5B [2]) across different attributes, the biases probably come from it. In order to solve the bias generated from the training dataset, we specifically focus the text-to-image stable diffusion models and define our problem as follow: Given a pre-trained Stable Diffusion model and an user specified text prompt, we want the number of generated images for each subclass across some attribute to be the same<\/em><\/strong>.<\/p>\n\n\n\n

\"\"
Figure 2. Statement of our problem definition across gender attributes.<\/em><\/figcaption><\/figure>\n\n\n\n

Some of the straightforward ideas to solve this problems are too difficult to implement, such as rebuilding the whole LAION-5B as a fair dataset, or retrain the Stable Diffusion, which will cost nearly 1 month using 200 GPUs. So, how are we going to solve this problem? Let’s move to the next Method<\/em><\/a><\/strong> section…<\/p>\n\n\n\n

References<\/h2>\n\n\n\n
  1. Jonathan Ho, et al. \u201cDenoising Diffusion Probabilistic Models.\u201d\u00a0NIPS<\/em>. 2020.<\/li>
  2. Schuhmann Christoph, et al. \u201cLaion-5b: An open large-scale dataset for training next generation image-text models\u201d arXiv. 2022<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"

    Motivation We are experiencing an exciting time of generative models. With the appearance and development of diffusion models [1], they are gradually replacing the prevalent generative adversarial network (GAN). Though astonishing and high-quality images are generated, there still leave many serious questions that need to be solved. Among them, fairness is a very crucial and … <\/p>\n

    Continue reading “Overview”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"","meta":{"footnotes":""},"class_list":["post-2","page","type-page","status-publish","hentry"],"yoast_head":"\nOverview - FairToken: Learning Fair Text Representations<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Overview - FairToken: Learning Fair Text Representations\" \/>\n<meta property=\"og:description\" content=\"Motivation We are experiencing an exciting time of generative models. With the appearance and development of diffusion models [1], they are gradually replacing the prevalent generative adversarial network (GAN). Though astonishing and high-quality images are generated, there still leave many serious questions that need to be solved. Among them, fairness is a very crucial and … Continue reading "Overview"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/\" \/>\n<meta property=\"og:site_name\" content=\"FairToken: Learning Fair Text Representations\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-20T03:37:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-content\/uploads\/sites\/73\/2022\/12\/img1.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/\",\"name\":\"Overview - FairToken: Learning Fair Text Representations\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/wp-content\\\/uploads\\\/sites\\\/73\\\/2022\\\/12\\\/img1.png\",\"datePublished\":\"2022-12-14T20:16:00+00:00\",\"dateModified\":\"2022-12-20T03:37:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/wp-content\\\/uploads\\\/sites\\\/73\\\/2022\\\/12\\\/img1.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/wp-content\\\/uploads\\\/sites\\\/73\\\/2022\\\/12\\\/img1.png\",\"width\":898,\"height\":442},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/\",\"name\":\"FairToken: Learning Fair Text Representations\",\"description\":\"Unbiased Text-to-Image Generative Models | Students: Siqi Chai, Xuanbai Chen | Advisor: Fernando De la Torre\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2023team2\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Overview - FairToken: Learning Fair Text Representations","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","og_locale":"en_US","og_type":"article","og_title":"Overview - FairToken: Learning Fair Text Representations","og_description":"Motivation We are experiencing an exciting time of generative models. With the appearance and development of diffusion models [1], they are gradually replacing the prevalent generative adversarial network (GAN). Though astonishing and high-quality images are generated, there still leave many serious questions that need to be solved. Among them, fairness is a very crucial and … Continue reading \"Overview\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","og_site_name":"FairToken: Learning Fair Text Representations","article_modified_time":"2022-12-20T03:37:47+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-content\/uploads\/sites\/73\/2022\/12\/img1.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","name":"Overview - FairToken: Learning Fair Text Representations","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-content\/uploads\/sites\/73\/2022\/12\/img1.png","datePublished":"2022-12-14T20:16:00+00:00","dateModified":"2022-12-20T03:37:47+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-content\/uploads\/sites\/73\/2022\/12\/img1.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-content\/uploads\/sites\/73\/2022\/12\/img1.png","width":898,"height":442},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/"},{"@type":"ListItem","position":2,"name":"Overview"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/","name":"FairToken: Learning Fair Text Representations","description":"Unbiased Text-to-Image Generative Models | Students: Siqi Chai, Xuanbai Chen | Advisor: Fernando De la Torre","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/pages\/2","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/comments?post=2"}],"version-history":[{"count":18,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/pages\/2\/revisions"}],"predecessor-version":[{"id":115,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/pages\/2\/revisions\/115"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2023team2\/wp-json\/wp\/v2\/media?parent=2"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}