{"id":63,"date":"2022-05-02T00:49:34","date_gmt":"2022-05-02T00:49:34","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/?page_id=63"},"modified":"2022-12-16T20:29:57","modified_gmt":"2022-12-16T20:29:57","slug":"classical-methods","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/","title":{"rendered":"Fast Depth Extraction"},"content":{"rendered":"\n<p>Please read <a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/\">Home<\/a> and <a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/ground-truth-estimation\/\">Ground Truth Estimation<\/a> before proceeding. <\/p>\n\n\n\n<p>The technical name for this method is <strong>Synthetic Wavelength Interferometry<\/strong>.  We aim to account for noise in the system using machine learning.<\/p>\n\n\n\n<p>We pass two wavelengths instead of a broadband source.  We get the intensity of pixel varying with mirror distance x like the white curve in Figure 1 .<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-style-default\"><img loading=\"lazy\" decoding=\"async\" width=\"853\" height=\"480\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif\" alt=\"\" class=\"wp-image-64\" \/><figcaption>Figure 1<\/figcaption><\/figure>\n\n\n\n<p>If you&#8217;re familiar with amplitude modulation, there is a carrier wave which is high frequency and there is a message wave (green curve in Figure 1) which defines the  amplitude. <\/p>\n\n\n\n<p>Note that this green curve i.e. the message peaks at <strong>x = d + (n * wavelength)<\/strong> where <strong>d<\/strong> is the depth and <strong>n<\/strong> is integer <\/p>\n\n\n\n<p>Using amplitude demodulation or carrier removal described below, we are able to get what measurements on message wave would look like. Assume for now, we need 4 measurements at  predefined locations on the message wave to get depth.<\/p>\n\n\n\n<p> Intuitively, to get 1 message wave measurement at some location, we  take 4 measurements close to each other around the desired location. The gap between the 4 measurements is such that we cover one oscillation of high frequency carrier wave. See Figure 2<\/p>\n\n\n\n<p>Since carrier is high frequency, we can assume the amplitude changes due to the message wave for let&#8217;s say the 4 blue measurements(in Figure 2) is negligible. We can just take the norm of the 4 measurements to get the amplitude.<\/p>\n\n\n\n<p>We take 16 measurements as shown in Figure 2 to get 4 message wave measurements. <\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-style-default\"><img loading=\"lazy\" decoding=\"async\" width=\"853\" height=\"480\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/carrier-removal.gif\" alt=\"\" class=\"wp-image-73\" \/><figcaption>Figure 2<\/figcaption><\/figure>\n\n\n\n<p>We now have 4 measurements of the message wave at our predefined locations. <\/p>\n\n\n\n<p>Assume through some math, preprocessing and defining our measurement locations intelligently, the 4 message wave measurements for a given pixel <strong>p<\/strong> for our purposes represent <strong>[cos(d), sin(d), -cos(d), -sin(d)]<\/strong> where <strong>d<\/strong> is <strong>depth*2pi \/wavelength<\/strong>. But you can think <strong>d<\/strong> is depth since scaling factor of <strong>depth<\/strong> is known and therefore irrelevant. And all depths for our analysis are angles which can be scaled using a known scale to get millimeters<\/p>\n\n\n\n<p>Note that we get same 4 measurements if <strong>d<\/strong> was <strong>d+2pi<\/strong> . This means we can only measure depth uniquely if we knew for a fact that our entire scene is between <strong>0<\/strong> and <strong>2pi<\/strong>. The reason my initial approaches in this project didn&#8217;t work were because this assumption of constraints on depth was not true for our dataset. While our scene had <strong>max depth &#8211; min depth  &lt;= 2pi<\/strong>, there was no guarantee that min depth wasn&#8217;t negative. Since we constrained any extracted depth to be between <strong>0<\/strong> to <strong>2pi<\/strong>, the negative <strong>d<\/strong>&#8216;s get mapped to <strong>2pi +d<\/strong> . This would also happen if max depth was greater than <strong>2pi<\/strong>.<\/p>\n\n\n\n<p> The above effect is called <em>phase wrapping<\/em>. The below image demonstrates it.<\/p>\n\n\n\n<figure class=\"wp-block-image is-style-default\"><img decoding=\"async\" src=\"https:\/\/www.mdpi.com\/sensors\/sensors-18-01793\/article_deploy\/html\/images\/sensors-18-01793-g001-550.jpg\" alt=\"Sensors 18 01793 g001 550\" \/><figcaption>(a) Depth image b) Phase wrapped image<\/figcaption><\/figure>\n\n\n\n<p>Note that in above example scene has much larger variations in depth that our dataset, thus more nested contours. <\/p>\n\n\n\n<p>Assume that there exists phase unwrapping algorithms to get depth from phase wrapped depth.   <\/p>\n\n\n\n<p>Our contribution in this work is as follows, we tried following approaches<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Training a model which takes measurements and gives us a depth . We  compare output depth to match ground truth depth.<\/li><li>We simply convert our ground truth depth to what the ideal 4 measurements would look like and train the model to denoise our measurements to match the ideal measurements. <\/li><\/ol>\n\n\n\n<p> The first model would have to model inverse trigonometry and then take care of modelling phase unwrapping whereas the second model only has to worry about removing noise in our sensor data. We can do inverse trigonometry and phase unwrapping on top of the clean measurements our model gives. <\/p>\n\n\n\n<p>A small detail in 2nd solution is to use a per-scene shift parameter for depth before converting ground truth depth to ideal measurements. This parameter is trainable and accounts for any shift in depth across the two methods. Another thing is we maximize the dot product of ideal measurements HxWx4 and clean measurements HxWx4 along the last dimension instead of taking an L2 loss. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Results(trained for very few epochs and small network)<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large is-style-default\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"436\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/12\/image-1024x436.png\" alt=\"\" class=\"wp-image-141\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/12\/image-1024x436.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/12\/image-300x128.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/12\/image-768x327.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/12\/image.png 1112w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p>Error RMSE<\/p>\n\n\n\n<p>Using noisy measurements  : 0.0314 mm<\/p>\n\n\n\n<p>Using clean measurements : 0.0257 mm<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Please read Home and Ground Truth Estimation before proceeding. The technical name for this method is Synthetic Wavelength Interferometry. We aim to account for noise in the system using machine learning. We pass two wavelengths instead of a broadband source. We get the intensity of pixel varying with mirror distance x like the white curve &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Fast Depth Extraction&#8221;<\/span><\/a><\/p>\n","protected":false},"author":130,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-63","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fast Depth Extraction - Neural Synthetic Wavelength Interferometry<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fast Depth Extraction - Neural Synthetic Wavelength Interferometry\" \/>\n<meta property=\"og:description\" content=\"Please read Home and Ground Truth Estimation before proceeding. The technical name for this method is Synthetic Wavelength Interferometry. We aim to account for noise in the system using machine learning. We pass two wavelengths instead of a broadband source. We get the intensity of pixel varying with mirror distance x like the white curve &hellip; Continue reading &quot;Fast Depth Extraction&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/\" \/>\n<meta property=\"og:site_name\" content=\"Neural Synthetic Wavelength Interferometry\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-16T20:29:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/\",\"name\":\"Fast Depth Extraction - Neural Synthetic Wavelength Interferometry\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/wp-content\\\/uploads\\\/sites\\\/60\\\/2022\\\/05\\\/syn-wavelength-4.gif\",\"datePublished\":\"2022-05-02T00:49:34+00:00\",\"dateModified\":\"2022-12-16T20:29:57+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/wp-content\\\/uploads\\\/sites\\\/60\\\/2022\\\/05\\\/syn-wavelength-4.gif\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/wp-content\\\/uploads\\\/sites\\\/60\\\/2022\\\/05\\\/syn-wavelength-4.gif\",\"width\":853,\"height\":480},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/classical-methods\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fast Depth Extraction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/\",\"name\":\"Neural Synthetic Wavelength Interferometry\",\"description\":\"Micron scale depth estimation\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2022team5\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fast Depth Extraction - Neural Synthetic Wavelength Interferometry","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/","og_locale":"en_US","og_type":"article","og_title":"Fast Depth Extraction - Neural Synthetic Wavelength Interferometry","og_description":"Please read Home and Ground Truth Estimation before proceeding. The technical name for this method is Synthetic Wavelength Interferometry. We aim to account for noise in the system using machine learning. We pass two wavelengths instead of a broadband source. We get the intensity of pixel varying with mirror distance x like the white curve &hellip; Continue reading \"Fast Depth Extraction\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/","og_site_name":"Neural Synthetic Wavelength Interferometry","article_modified_time":"2022-12-16T20:29:57+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/","name":"Fast Depth Extraction - Neural Synthetic Wavelength Interferometry","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif","datePublished":"2022-05-02T00:49:34+00:00","dateModified":"2022-12-16T20:29:57+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-content\/uploads\/sites\/60\/2022\/05\/syn-wavelength-4.gif","width":853,"height":480},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/classical-methods\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/"},{"@type":"ListItem","position":2,"name":"Fast Depth Extraction"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/","name":"Neural Synthetic Wavelength Interferometry","description":"Micron scale depth estimation","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/pages\/63","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/users\/130"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/comments?post=63"}],"version-history":[{"count":22,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/pages\/63\/revisions"}],"predecessor-version":[{"id":151,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/pages\/63\/revisions\/151"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2022team5\/wp-json\/wp\/v2\/media?parent=63"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}