{"id":18,"date":"2020-05-10T05:01:00","date_gmt":"2020-05-10T05:01:00","guid":{"rendered":"http:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/?page_id=18"},"modified":"2020-12-17T19:40:55","modified_gmt":"2020-12-17T19:40:55","slug":"introduction","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","title":{"rendered":"Overview"},"content":{"rendered":"\n
\"\"
High level idea here we try<\/figcaption><\/figure>\n\n\n\n

FRL Pittsburgh is on a quest to build photorealistic face avatars from high quality capture studio data, aiming to appear as close to real life as possible. However, the real use of these avatars is not simply reconstructing those high quality data, but being animated in realtime from limited sensors mounted on VR headset at the same fidelity, to make social interactions in VR believable. The goal of this project is to tackle the following two main challenges: <\/p>

(1) these sensors only see non-overlapping patches from the cameras very close to face in different spectrum (infrared)<\/p>

(2) the built system should be robust to variations of input, caused by factors such as slightly different camera positions VR headset, user’s headset wearing positions on the face, and lighting\/background variations in the environment.<\/p><\/blockquote>\n\n\n\n

Problem Statement<\/h2>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

You can see our methods and results in the semester wise pages https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/spring-semester\/<\/a><\/p>\n\n\n\n

https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/fall-semester-2\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

FRL Pittsburgh is on a quest to build photorealistic face avatars from high quality capture studio data, aiming to appear as close to real life as possible. However, the real use of these avatars is not simply reconstructing those high quality data, but being animated in realtime from limited sensors mounted on VR headset at … <\/p>\n

Continue reading “Overview”<\/span><\/a><\/p>\n","protected":false},"author":94,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-18","page","type-page","status-publish","hentry"],"yoast_head":"\nOverview - Real-time Photorealistic VR Facial Animation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Overview - Real-time Photorealistic VR Facial Animation\" \/>\n<meta property=\"og:description\" content=\"FRL Pittsburgh is on a quest to build photorealistic face avatars from high quality capture studio data, aiming to appear as close to real life as possible. However, the real use of these avatars is not simply reconstructing those high quality data, but being animated in realtime from limited sensors mounted on VR headset at … Continue reading "Overview"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/\" \/>\n<meta property=\"og:site_name\" content=\"Real-time Photorealistic VR Facial Animation\" \/>\n<meta property=\"article:modified_time\" content=\"2020-12-17T19:40:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/12\/Screenshot-2020-12-16-at-3.39.40-PM.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1852\" \/>\n\t<meta property=\"og:image:height\" content=\"990\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\",\"name\":\"Overview - Real-time Photorealistic VR Facial Animation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/wp-content\\\/uploads\\\/sites\\\/43\\\/2020\\\/12\\\/Screenshot-2020-12-16-at-3.39.40-PM-1024x547.png\",\"datePublished\":\"2020-05-10T05:01:00+00:00\",\"dateModified\":\"2020-12-17T19:40:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/wp-content\\\/uploads\\\/sites\\\/43\\\/2020\\\/12\\\/Screenshot-2020-12-16-at-3.39.40-PM.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/wp-content\\\/uploads\\\/sites\\\/43\\\/2020\\\/12\\\/Screenshot-2020-12-16-at-3.39.40-PM.png\",\"width\":1852,\"height\":990},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\",\"name\":\"Real-time Photorealistic VR Facial Animation\",\"description\":\"Sponsored by Facebook Reality Labs\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#organization\",\"name\":\"Real-time Photorealistic VR Facial Animation\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/wp-content\\\/uploads\\\/sites\\\/43\\\/2020\\\/05\\\/cropped-cmu-log.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/wp-content\\\/uploads\\\/sites\\\/43\\\/2020\\\/05\\\/cropped-cmu-log.png\",\"width\":250,\"height\":250,\"caption\":\"Real-time Photorealistic VR Facial Animation\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2020teamm\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Overview - Real-time Photorealistic VR Facial Animation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","og_locale":"en_US","og_type":"article","og_title":"Overview - Real-time Photorealistic VR Facial Animation","og_description":"FRL Pittsburgh is on a quest to build photorealistic face avatars from high quality capture studio data, aiming to appear as close to real life as possible. However, the real use of these avatars is not simply reconstructing those high quality data, but being animated in realtime from limited sensors mounted on VR headset at … Continue reading \"Overview\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","og_site_name":"Real-time Photorealistic VR Facial Animation","article_modified_time":"2020-12-17T19:40:55+00:00","og_image":[{"width":1852,"height":990,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/12\/Screenshot-2020-12-16-at-3.39.40-PM.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","name":"Overview - Real-time Photorealistic VR Facial Animation","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/12\/Screenshot-2020-12-16-at-3.39.40-PM-1024x547.png","datePublished":"2020-05-10T05:01:00+00:00","dateModified":"2020-12-17T19:40:55+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/12\/Screenshot-2020-12-16-at-3.39.40-PM.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/12\/Screenshot-2020-12-16-at-3.39.40-PM.png","width":1852,"height":990},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/"},{"@type":"ListItem","position":2,"name":"Overview"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","name":"Real-time Photorealistic VR Facial Animation","description":"Sponsored by Facebook Reality Labs","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#organization","name":"Real-time Photorealistic VR Facial Animation","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/05\/cropped-cmu-log.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-content\/uploads\/sites\/43\/2020\/05\/cropped-cmu-log.png","width":250,"height":250,"caption":"Real-time Photorealistic VR Facial Animation"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/pages\/18","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/users\/94"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/comments?post=18"}],"version-history":[{"count":10,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/pages\/18\/revisions"}],"predecessor-version":[{"id":174,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/pages\/18\/revisions\/174"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2020teamm\/wp-json\/wp\/v2\/media?parent=18"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}