{"id":116,"date":"2023-12-07T06:26:10","date_gmt":"2023-12-07T06:26:10","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/?page_id=116"},"modified":"2023-12-07T07:27:43","modified_gmt":"2023-12-07T07:27:43","slug":"fall-23-progress","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/","title":{"rendered":"Fall &#8217;23 Progress"},"content":{"rendered":"\n<h4 class=\"wp-block-heading\">Resources<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.google.com\/presentation\/d\/1LwxAO5nKST9QtQUuwY5MIUkzivl-rVMlaIac-shDByM\/edit#slide=id.p\">Presentation slides<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/docs.google.com\/presentation\/d\/14-qk4OWwuhfBJtR8whmGB1s1oW64cA1OTklnxl2jVFo\/edit?usp=sharing\">Poster<\/a><\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Unsupervised Model Diagnosis (UMO)<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We propose an unsupervised framework for model diagnosis, named UMO, that bypasses the tedious and expensive requirement of human annotation and user inputs.<\/li>\n\n\n\n<li>UMO utilizes the parametric knowledge from foundation models to ensure accurate analysis of model vulnerabilities. The framework does not require a manually-defined list of attributes to generate counterfactual examples.<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"816\" height=\"538\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.34.56\u202fAM.png\" alt=\"\" class=\"wp-image-122\" style=\"width:650px;height:auto\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.34.56\u202fAM.png 816w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.34.56\u202fAM-300x198.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.34.56\u202fAM-768x506.png 768w\" sizes=\"auto, (max-width: 816px) 100vw, 816px\" \/><\/figure>\n\n\n\n<h5 class=\"wp-block-heading\">Method Overview<\/h5>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"301\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM-1024x301.png\" alt=\"\" class=\"wp-image-121\" style=\"width:650px;height:auto\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM-1024x301.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM-300x88.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM-768x226.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM-1536x451.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.33.23\u202fAM.png 1668w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Counterfactual Optimization: <\/strong>directly optimizing the latent edit directions that can mislead prediction of target model<\/li>\n\n\n\n<li><strong>Counterfactual Analysis: <\/strong>generate pairs of (original, counterfactual) to discover semantic attributes<\/li>\n<\/ol>\n\n\n\n<h5 class=\"wp-block-heading\">Experiments<\/h5>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment: Generate counterfactuals for target models of known biases<\/h6>\n\n\n\n<p>This section evaluates UMO through experiments with classifiers trained on imbalanced data. <\/p>\n\n\n\n<p>Our experiments consistently highlight that the diagnosis from UMO can reliably pinpoint these intentional biases, confirming the reliability of our unsupervised detection.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"452\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1024x452.jpg\" alt=\"\" class=\"wp-image-123\" style=\"width:650px;height:auto\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1024x452.jpg 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-300x132.jpg 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-768x339.jpg 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1536x678.jpg 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled.jpg 1754w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"850\" height=\"472\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.41\u202fAM.png\" alt=\"\" class=\"wp-image-124\" style=\"width:650px;height:auto\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.41\u202fAM.png 850w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.41\u202fAM-300x167.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.41\u202fAM-768x426.png 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/figure>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment: Cross-Method Diagnosis Consistency<\/h6>\n\n\n\n<p>We performed three distinct experiments: two using our framework with different generative model backbones (Diffusion model and StyleGAN), and the other using the ZOOM approach. We should expect to discover the same counterfactual attributes for a given target model, across all three experiments.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"842\" height=\"476\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.55\u202fAM.png\" alt=\"\" class=\"wp-image-125\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.55\u202fAM.png 842w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.55\u202fAM-300x170.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.44.55\u202fAM-768x434.png 768w\" sizes=\"auto, (max-width: 842px) 100vw, 842px\" \/><\/figure>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment: Generalization to Other vision Tasks<\/h6>\n\n\n\n<p>We expanded our experiments to encompass image segmentation and keypoint detection tasks. As shown, we are able to successfully attack the target model using our method.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"242\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM-1024x242.png\" alt=\"\" class=\"wp-image-126\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM-1024x242.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM-300x71.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM-768x182.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM-1536x363.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-1.45.09\u202fAM.png 1784w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading has-large-font-size\">Domain Gap Embeddings (DoGE)<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We introduce Domain Gap Embeddings (DoGE), a plug-and-play semantic data augmentation framework in a cross-distribution few-shot setting. <\/li>\n\n\n\n<li>Our method extracts disparities between source and desired data distributions in a latent form, and subsequently steers a generative process to supplement the training set with endless diverse synthetic samples.<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"727\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.03.07-AM-1024x727.png\" alt=\"\" class=\"wp-image-130\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.03.07-AM-1024x727.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.03.07-AM-300x213.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.03.07-AM-768x545.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.03.07-AM.png 1310w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h5 class=\"wp-block-heading\">Method overview<\/h5>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"296\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM-1024x296.png\" alt=\"\" class=\"wp-image-131\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM-1024x296.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM-300x87.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM-768x222.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM-1536x443.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.09.35-AM.png 1888w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Encode a few data samples from source\/target into the CLIP embedding space<\/li>\n\n\n\n<li>Extract the Domain Gap Embedding (\u2206)<\/li>\n\n\n\n<li>Augment source data embeddings with\u00a0\u2206<\/li>\n\n\n\n<li>Generate target images from the augmented embeddings with Stable UnCLIP<\/li>\n<\/ol>\n\n\n\n<h5 class=\"wp-block-heading\">experiments<\/h5>\n\n\n\n<p>This section evaluates DoGE through experiments with classifiers trained on a combination of real and synthetic data.<\/p>\n\n\n\n<p>Our evaluations, conducted on a subpopulation shift and three domain adaptation scenarios under a few-shot paradigm, reveal that our versatile method improves performance across tasks without needing hands-on intervention or intricate fine-tuning.<\/p>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment: mitigating Subpopulation shift<\/h6>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"242\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM-1024x242.png\" alt=\"\" class=\"wp-image-133\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM-1024x242.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM-300x71.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM-768x182.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM-1536x363.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.42-AM.png 1902w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Examples of synthetic CelebA data generated from (a) Source into (b) Target distribution. Under subpopulation shift, we generated data from the majority subpopulation (a) into the under-represented distribution (b). (c) shows the synthetic data generated from our pipeline. The results demonstrate our capability to apply semantic augmentation in accordance with gaps between two distributions.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"686\" height=\"364\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.20-AM.png\" alt=\"\" class=\"wp-image-132\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.20-AM.png 686w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.18.20-AM-300x159.png 300w\" sizes=\"auto, (max-width: 686px) 100vw, 686px\" \/><\/figure>\n\n\n\n<p>Test Accuracy on our constructed CelebA imbalanced classification problem. We evaluated our method against four baselines. This table shows that synthetic data from DoGE has a significant advantage over other methods.<\/p>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment on unsupervised domain adaptation problem (classification)<\/h6>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"271\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM-1024x271.png\" alt=\"\" class=\"wp-image-134\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM-1024x271.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM-300x79.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM-768x203.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM-1536x406.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.22.19-AM.png 1870w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Examples of synthetic DomainNet data, generated from source data into four different target domains. Each generation (bottom) was augmented from the source image (top) using our pipeline with ControlNet. The results demonstrate our capability to augment data in accordance with gaps between distributions.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"878\" height=\"502\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.41-AM.png\" alt=\"\" class=\"wp-image-137\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.41-AM.png 878w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.41-AM-300x172.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.41-AM-768x439.png 768w\" sizes=\"auto, (max-width: 878px) 100vw, 878px\" \/><\/figure>\n\n\n\n<p>Examples of synthetic FMoW data, generated from Source (2002-12) into Target (2016-17) distributions in 3 regions using our method with ControlNet. The results illustrate our capacity to generate images across temporal discrepancies.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"264\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM-1024x264.png\" alt=\"\" class=\"wp-image-135\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM-1024x264.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM-300x77.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM-768x198.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM-1536x396.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.23.14-AM.png 1542w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Test accuracy in unsupervised domain adaptation classification problems. We evaluated against four baselines on the left column. For DomainNet, the task is to adopt a model with a Real domain training dataset to Painting, Inforgraph, Clipart, and Sketch domains. For FMoW, for each region (Asia, Americas, Oceania), we adopted a model with old satellite images (2002-15) to perform well on new satellite data (2016-18). The table shows that our methods achieved the highest test accuracy in every category.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"812\" height=\"422\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.20-AM.png\" alt=\"\" class=\"wp-image-136\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.20-AM.png 812w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.20-AM-300x156.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.24.20-AM-768x399.png 768w\" sizes=\"auto, (max-width: 812px) 100vw, 812px\" \/><\/figure>\n\n\n\n<p>Test Accuracy of UDA methods on the DomainNet (Real \u2192 Painting) problem. We evaluated existing UDA methods with and without DoGE. The table shows that our approach is compatible with and complementary to UDA methods.<\/p>\n\n\n\n<h6 class=\"wp-block-heading\">Experiment on Unsupervised domain adaptation problem (segmentation)<\/h6>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"247\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM-1024x247.png\" alt=\"\" class=\"wp-image-138\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM-1024x247.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM-300x72.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM-768x185.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM-1536x371.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.26.42-AM.png 1880w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Examples of synthetic self-driving data generated from (a) GTA5 source images into (b) Cityscapes target domain. (c) shows the synthetic data generated from our pipeline without any improvement tricks. We also demonstrated the generation with scene structure preserved by ControlNet (conditioned on canny edges and source segmentation ground truth) in (d). The synthetic data are then used in unsupervised domain adaptation methods to adapt models across domains.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"806\" height=\"276\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.27.16-AM.png\" alt=\"\" class=\"wp-image-139\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.27.16-AM.png 806w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.27.16-AM-300x103.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Screenshot-2023-12-07-at-2.27.16-AM-768x263.png 768w\" sizes=\"auto, (max-width: 806px) 100vw, 806px\" \/><\/figure>\n\n\n\n<p>GTA5 \u2192 Cityscapes cross-domain segmentation. We used DAFormer as our UDA baseline. DATUM and DoGE are target data generators applied on top of DAFormer. Our performance is at par with DATUM while exempt from any training.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Resources Unsupervised Model Diagnosis (UMO) Method Overview Experiments Experiment: Generate counterfactuals for target models of known biases This section evaluates UMO through experiments with classifiers trained on imbalanced data. Our experiments consistently highlight that the diagnosis from UMO can reliably pinpoint these intentional biases, confirming the reliability of our unsupervised detection. Experiment: Cross-Method Diagnosis Consistency [&hellip;]<\/p>\n","protected":false},"author":172,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-116","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fall &#039;23 Progress - Model Diagnosis with Domain Adaptation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fall &#039;23 Progress - Model Diagnosis with Domain Adaptation\" \/>\n<meta property=\"og:description\" content=\"Resources Unsupervised Model Diagnosis (UMO) Method Overview Experiments Experiment: Generate counterfactuals for target models of known biases This section evaluates UMO through experiments with classifiers trained on imbalanced data. Our experiments consistently highlight that the diagnosis from UMO can reliably pinpoint these intentional biases, confirming the reliability of our unsupervised detection. Experiment: Cross-Method Diagnosis Consistency [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/\" \/>\n<meta property=\"og:site_name\" content=\"Model Diagnosis with Domain Adaptation\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-07T07:27:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1024x452.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/\",\"name\":\"Fall '23 Progress - Model Diagnosis with Domain Adaptation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/wp-content\\\/uploads\\\/sites\\\/87\\\/2023\\\/12\\\/Untitled-1024x452.jpg\",\"datePublished\":\"2023-12-07T06:26:10+00:00\",\"dateModified\":\"2023-12-07T07:27:43+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/wp-content\\\/uploads\\\/sites\\\/87\\\/2023\\\/12\\\/Untitled.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/wp-content\\\/uploads\\\/sites\\\/87\\\/2023\\\/12\\\/Untitled.jpg\",\"width\":1754,\"height\":774},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/fall-23-progress\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fall &#8217;23 Progress\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/\",\"name\":\"Model Diagnosis with Domain Adaptation\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team10\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fall '23 Progress - Model Diagnosis with Domain Adaptation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/","og_locale":"en_US","og_type":"article","og_title":"Fall '23 Progress - Model Diagnosis with Domain Adaptation","og_description":"Resources Unsupervised Model Diagnosis (UMO) Method Overview Experiments Experiment: Generate counterfactuals for target models of known biases This section evaluates UMO through experiments with classifiers trained on imbalanced data. Our experiments consistently highlight that the diagnosis from UMO can reliably pinpoint these intentional biases, confirming the reliability of our unsupervised detection. Experiment: Cross-Method Diagnosis Consistency [&hellip;]","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/","og_site_name":"Model Diagnosis with Domain Adaptation","article_modified_time":"2023-12-07T07:27:43+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1024x452.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/","name":"Fall '23 Progress - Model Diagnosis with Domain Adaptation","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled-1024x452.jpg","datePublished":"2023-12-07T06:26:10+00:00","dateModified":"2023-12-07T07:27:43+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-content\/uploads\/sites\/87\/2023\/12\/Untitled.jpg","width":1754,"height":774},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/fall-23-progress\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/"},{"@type":"ListItem","position":2,"name":"Fall &#8217;23 Progress"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/","name":"Model Diagnosis with Domain Adaptation","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/pages\/116","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/users\/172"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/comments?post=116"}],"version-history":[{"count":3,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/pages\/116\/revisions"}],"predecessor-version":[{"id":140,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/pages\/116\/revisions\/140"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team10\/wp-json\/wp\/v2\/media?parent=116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}