{"id":33,"date":"2023-05-08T18:10:32","date_gmt":"2023-05-08T18:10:32","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/?page_id=33"},"modified":"2023-05-09T15:39:28","modified_gmt":"2023-05-09T15:39:28","slug":"overview","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","title":{"rendered":"Overview"},"content":{"rendered":"\n<p><\/p>\n\n\n\n<p class=\"has-text-align-center\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048\" style=\"width: 1000px\"><\/p>\n\n\n\n<p>Mugsy is a High-end Multi-view Capturing System that helps capture synchronized multi-view videos of facial expressions exhibited by subjects inside the dome. The result of this large-scale camera setup is the dataset, Multiface. It consists of high-quality recordings of the faces of 13 identities, each captured in a multi-view capture stage performing various facial expressions. An average of 12,200 (left) to 23,000 (right) frames per subject were captured at 30 fps. <\/p>\n\n\n\n<p><strong>Applications<\/strong>: Photo-realistic human face rendering and reconstruction<\/p>\n\n\n\n<p class=\"has-text-align-center\"><img decoding=\"async\" src=\"https:\/\/lh5.googleusercontent.com\/aQZrSW7znoGuv4wI5pfEGNeggrXd1iPwhPcguO2qiZKQ8lac47ioq0wrvCHR9mf5ihDrSBT78FONA3sBz_wmnNXY6xwkVo964EodVu9xIKMSLTFkxflpEXVHWempQqnJJlsxbewDmVTKyXl1Gm9rI3CBUA=s2048\" style=\"width: 400px\"><\/p>\n\n\n\n<p>However, calibrating the cameras on such a large-scale using external calibration objects is time-consuming. Currently, these images are calibrated using an icosahedron with deltille grids pasted on top of its faces. <\/p>\n\n\n\n<p class=\"has-text-align-center\"><img decoding=\"async\" src=\"https:\/\/lh6.googleusercontent.com\/ogtzb7Gqw4SxyptcLTLxhvTFlMlxqfRfevqdSg9rx7jIEgQGUsCdLxiTZCac7zhcuuqNQNGEkt0CtfWCjA7BeXwIxkU7_6HB2cr3_oUFepflL9f7HeWTUDZ9cQBb1afu6MRKusyB50XkU90L-ZZ6WtjLFw=s2048\" style=\"width: 200px\"><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity is-style-wide\" \/>\n\n\n\n<h1 class=\"wp-block-heading\"><strong>Problem Statement<\/strong><\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Build an <strong>auto-calibration system<\/strong> capable of determining internal camera parameters directly from multiple uncalibrated images.&nbsp;It is assumed that the intrinsic parameters of the camera change with respect to the subject, however the extrinsic, i.e., the position and locations of the cameras in the dome do not undergo major changes. <\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Integrate the above system into the <strong>Structure from Motion (SfM)<\/strong> pipeline (presently, we are using Colmap) to achieve better calibration precision from various optimization tricks (eg. bundle adjustment) when compared to computing calibration data using a special calibration object.<\/li>\n<\/ul>\n\n\n\n<p>We plan to use only the data that is available to us (multi-view capture raw image data) to determine the intrinsic parameters of the large-scale camera setup of a mugsy. <\/p>\n\n\n\n<p><img decoding=\"async\" width=\"1051px;\" height=\"469px;\" src=\"https:\/\/lh5.googleusercontent.com\/f7IXjDU_tNiaWBhpGODxjJ4lVGxfNubS3TtcDs06KaJ7WDNCxevG6OM69hW2_lTe9OAXaEdXVfcbLkUDnRQtYnSHd8bpe7yIspqRwlSyf_3m3WNsWMSld9ejAvE9eZ08NUWve6sr-_7KBouTPzIDcsrZ7w=s2048\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Mugsy is a High-end Multi-view Capturing System that helps capture synchronized multi-view videos of facial expressions exhibited by subjects inside the dome. The result of this large-scale camera setup is the dataset, Multiface. It consists of high-quality recordings of the faces of 13 identities, each captured in a multi-view capture stage performing various facial expressions. &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Overview&#8221;<\/span><\/a><\/p>\n","protected":false},"author":159,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-33","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Overview - Large Scale Camera Array Calibration via SfM<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Overview - Large Scale Camera Array Calibration via SfM\" \/>\n<meta property=\"og:description\" content=\"Mugsy is a High-end Multi-view Capturing System that helps capture synchronized multi-view videos of facial expressions exhibited by subjects inside the dome. The result of this large-scale camera setup is the dataset, Multiface. It consists of high-quality recordings of the faces of 13 identities, each captured in a multi-view capture stage performing various facial expressions. &hellip; Continue reading &quot;Overview&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/\" \/>\n<meta property=\"og:site_name\" content=\"Large Scale Camera Array Calibration via SfM\" \/>\n<meta property=\"article:modified_time\" content=\"2023-05-09T15:39:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\",\"name\":\"Overview - Large Scale Camera Array Calibration via SfM\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/lh6.googleusercontent.com\\\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048\",\"datePublished\":\"2023-05-08T18:10:32+00:00\",\"dateModified\":\"2023-05-09T15:39:28+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#primaryimage\",\"url\":\"https:\\\/\\\/lh6.googleusercontent.com\\\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048\",\"contentUrl\":\"https:\\\/\\\/lh6.googleusercontent.com\\\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\",\"name\":\"Large Scale Camera Array Calibration via SfM\",\"description\":\"Students: Gaini Kussainova, Sanjana Gunna | Advisors: Wei Pu, Shubham Garg (Meta), Aswin Sankaranarayanan (CMU) | Sponsor: Meta\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#organization\",\"name\":\"Large Scale Camera Array Calibration via SfM\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/wp-content\\\/uploads\\\/sites\\\/80\\\/2023\\\/05\\\/meta.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/wp-content\\\/uploads\\\/sites\\\/80\\\/2023\\\/05\\\/meta.png\",\"width\":225,\"height\":225,\"caption\":\"Large Scale Camera Array Calibration via SfM\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/f23team3\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Overview - Large Scale Camera Array Calibration via SfM","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","og_locale":"en_US","og_type":"article","og_title":"Overview - Large Scale Camera Array Calibration via SfM","og_description":"Mugsy is a High-end Multi-view Capturing System that helps capture synchronized multi-view videos of facial expressions exhibited by subjects inside the dome. The result of this large-scale camera setup is the dataset, Multiface. It consists of high-quality recordings of the faces of 13 identities, each captured in a multi-view capture stage performing various facial expressions. &hellip; Continue reading \"Overview\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","og_site_name":"Large Scale Camera Array Calibration via SfM","article_modified_time":"2023-05-09T15:39:28+00:00","og_image":[{"url":"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","name":"Overview - Large Scale Camera Array Calibration via SfM","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#primaryimage"},"thumbnailUrl":"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048","datePublished":"2023-05-08T18:10:32+00:00","dateModified":"2023-05-09T15:39:28+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#primaryimage","url":"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048","contentUrl":"https:\/\/lh6.googleusercontent.com\/lHfLS5N4iVkoenU7HOM0upC4TewF3rldrCvuWa9WBgLhq_tWl8gty6R1XziYK1cxwLCreALKoL33NzVbMcTngkj38YeNHyVGi6NAttIMS7ZtIBrZ2xbpTnI4UHkq3HyL4O5RzCCcq3-JXMeNo6lEKYPt-Q=s2048"},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/"},{"@type":"ListItem","position":2,"name":"Overview"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","name":"Large Scale Camera Array Calibration via SfM","description":"Students: Gaini Kussainova, Sanjana Gunna | Advisors: Wei Pu, Shubham Garg (Meta), Aswin Sankaranarayanan (CMU) | Sponsor: Meta","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#organization","name":"Large Scale Camera Array Calibration via SfM","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-content\/uploads\/sites\/80\/2023\/05\/meta.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-content\/uploads\/sites\/80\/2023\/05\/meta.png","width":225,"height":225,"caption":"Large Scale Camera Array Calibration via SfM"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/pages\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/users\/159"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/comments?post=33"}],"version-history":[{"count":8,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/pages\/33\/revisions"}],"predecessor-version":[{"id":183,"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/pages\/33\/revisions\/183"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/f23team3\/wp-json\/wp\/v2\/media?parent=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}