{"id":40,"date":"2019-12-12T21:19:29","date_gmt":"2019-12-12T21:19:29","guid":{"rendered":"http:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/?page_id=40"},"modified":"2019-12-17T20:49:49","modified_gmt":"2019-12-17T20:49:49","slug":"project-summary","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","title":{"rendered":"Project summary"},"content":{"rendered":"

Motivation<\/strong><\/p>\n

What is the motivation of our project?<\/p>\n

People may unconsciously use their hands to express themselves during the conversation, and that is because hands are surprisingly important to convey information in communications. Although\u00a0the Facebook Reality Lab can now generate realistic facial reconstruction for virtual communications, it would be very beneficial to reconstruct hands for virtual human interaction and it would be much more natural to see a talking floating head along with two hands.<\/p>\n

On the other hand, it is worth exploration since it is more technically challenging compared to body or face tracking, due to the heavy occlusion and deformation.<\/p>\n

Problem<\/strong><\/p>\n

Our task is defined as multiview 3D pose estimation. The internal dataset Facebook reality lab gave us is called InterHand, which consists sequences of hand images captured from multiple cameras in multiple views. To further prove our method is effective, we also conducted experiments on Human3.6M.<\/p>\n

Solution<\/strong><\/p>\n

We firstly extract features and detect keypoints in 2D images. With camera information, we lift 2D keypoints to 3D. To further improve the 2D features, we propose “Epipolar Transformer” to utilize the epipolar geometry.<\/p>\n

Here is a 2 min video<\/a> to summarized our method and results.<\/p>\n

For more technical details, please check\u00a0Technical report<\/a> page and Video demostrations<\/a> page.<\/p>\n","protected":false},"excerpt":{"rendered":"

Motivation What is the motivation of our project? People may unconsciously use their hands to express themselves during the conversation, and that is because hands are surprisingly important to convey information in communications. Although\u00a0the Facebook Reality Lab can now generate realistic facial reconstruction for virtual communications, it would be very beneficial to reconstruct hands for … <\/p>\n

Continue reading “Project summary”<\/span><\/a><\/p>\n","protected":false},"author":39,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-40","page","type-page","status-publish","hentry"],"yoast_head":"\nProject summary - Tracking Two-Hand Interactions<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Project summary - Tracking Two-Hand Interactions\" \/>\n<meta property=\"og:description\" content=\"Motivation What is the motivation of our project? People may unconsciously use their hands to express themselves during the conversation, and that is because hands are surprisingly important to convey information in communications. Although\u00a0the Facebook Reality Lab can now generate realistic facial reconstruction for virtual communications, it would be very beneficial to reconstruct hands for … Continue reading "Project summary"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/\" \/>\n<meta property=\"og:site_name\" content=\"Tracking Two-Hand Interactions\" \/>\n<meta property=\"article:modified_time\" content=\"2019-12-17T20:49:49+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/\",\"name\":\"Project summary - Tracking Two-Hand Interactions\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/#website\"},\"datePublished\":\"2019-12-12T21:19:29+00:00\",\"dateModified\":\"2019-12-17T20:49:49+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Project summary\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/\",\"name\":\"Tracking Two-Hand Interactions\",\"description\":\"Sponsored by Facebook Reality Lab\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamc\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Project summary - Tracking Two-Hand Interactions","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","og_locale":"en_US","og_type":"article","og_title":"Project summary - Tracking Two-Hand Interactions","og_description":"Motivation What is the motivation of our project? People may unconsciously use their hands to express themselves during the conversation, and that is because hands are surprisingly important to convey information in communications. Although\u00a0the Facebook Reality Lab can now generate realistic facial reconstruction for virtual communications, it would be very beneficial to reconstruct hands for … Continue reading \"Project summary\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","og_site_name":"Tracking Two-Hand Interactions","article_modified_time":"2019-12-17T20:49:49+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","name":"Project summary - Tracking Two-Hand Interactions","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/#website"},"datePublished":"2019-12-12T21:19:29+00:00","dateModified":"2019-12-17T20:49:49+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/"},{"@type":"ListItem","position":2,"name":"Project summary"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/","name":"Tracking Two-Hand Interactions","description":"Sponsored by Facebook Reality Lab","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/pages\/40","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/comments?post=40"}],"version-history":[{"count":5,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/pages\/40\/revisions"}],"predecessor-version":[{"id":96,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/pages\/40\/revisions\/96"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamc\/wp-json\/wp\/v2\/media?parent=40"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}