{"id":37,"date":"2025-05-08T20:19:21","date_gmt":"2025-05-08T20:19:21","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/?page_id=37"},"modified":"2025-12-13T00:00:02","modified_gmt":"2025-12-13T00:00:02","slug":"next-steps","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/","title":{"rendered":"Future works"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Augment training images using diffusion models<\/h2>\n\n\n\n<p>While 360\u00b0 video capture provides dense multi-view imagery as the photographer moves through the scene, this walking-based capture results in input images being constrained to a nearly constant elevation. Although this is sufficient for training views along the capture path, geometric consistency, particularly for tall structures and distant objects, degrades as viewpoints move farther away from the trajectory.<\/p>\n\n\n\n<p>One way to address this limitation is to synthesize additional training views at a fixed elevated height (e.g., +1.5\u20132 m) offset from the original path, and refine these views using a diffusion-based model such as <strong>Difix3D+<\/strong>. These synthesized views can be incorporated into training to increase dataset coverage, especially in regions that are sparsely observed in the original capture.<\/p>\n\n\n\n<p>The same diffusion-based approach can also be employed at inference time as a learned rasterizer to suppress Gaussian Splatting artifacts in real-time novel view synthesis, further improving visual fidelity.<\/p>\n\n\n\n<p>Here are some examples of using Difix3D+ to improve visual fidelity and details in the novel view renders. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"331\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4-1024x331.png\" alt=\"\" class=\"wp-image-221\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4-1024x331.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4-300x97.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4-768x248.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4.png 1388w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"339\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5-1024x339.png\" alt=\"\" class=\"wp-image-223\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5-1024x339.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5-300x99.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5-768x254.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5-1536x508.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-5.png 1662w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h3 class=\"wp-block-heading\">Semantic Filtering<\/h3>\n\n\n\n<p>We can further improve reconstruction quality by removing dynamic distractors through semantic segmentation. Using equirectangular instance segmentation\u2014or, alternatively, performing segmentation on cubemap projections\u2014we generate masks for pedestrians, vehicles, and animals, and exclude these regions during Gaussian Splatting scene reconstruction.<\/p>\n\n\n\n<p>The examples below illustrate how such dynamic distractors introduce artifacts and degrade geometric consistency in GS reconstructions when left unfiltered.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"714\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-7-1024x714.png\" alt=\"\" class=\"wp-image-226\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-7-1024x714.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-7-300x209.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-7-768x535.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-7.png 1148w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h3 class=\"wp-block-heading\">Adaptive Sampling<\/h3>\n\n\n\n<p>To optimize efficiency during training and evaluation, we could implement <strong>adaptive subsampling strategies<\/strong> guided by the confidence maps. Instead of using all frames uniformly, the system will <strong>prioritize high-confidence, spatially diverse views<\/strong>, reducing redundancy and computational overhead without sacrificing reconstruction quality.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Augment training images using diffusion models While 360\u00b0 video capture provides dense multi-view imagery as the photographer moves through the scene, this walking-based capture results in input images being constrained to a nearly constant elevation. Although this is sufficient for training views along the capture path, geometric consistency, particularly for tall structures and distant objects, &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Future works&#8221;<\/span><\/a><\/p>\n","protected":false},"author":233,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-37","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting\" \/>\n<meta property=\"og:description\" content=\"Augment training images using diffusion models While 360\u00b0 video capture provides dense multi-view imagery as the photographer moves through the scene, this walking-based capture results in input images being constrained to a nearly constant elevation. Although this is sufficient for training views along the capture path, geometric consistency, particularly for tall structures and distant objects, &hellip; Continue reading &quot;Future works&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/\" \/>\n<meta property=\"og:site_name\" content=\"Adaptive Data Collection For High Fidelity Gaussian Splatting\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-13T00:00:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1388\" \/>\n\t<meta property=\"og:image:height\" content=\"448\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/\",\"name\":\"Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/wp-content\\\/uploads\\\/sites\\\/120\\\/2025\\\/12\\\/image-4-1024x331.png\",\"datePublished\":\"2025-05-08T20:19:21+00:00\",\"dateModified\":\"2025-12-13T00:00:02+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/wp-content\\\/uploads\\\/sites\\\/120\\\/2025\\\/12\\\/image-4.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/wp-content\\\/uploads\\\/sites\\\/120\\\/2025\\\/12\\\/image-4.png\",\"width\":1388,\"height\":448},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/next-steps\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Future works\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/\",\"name\":\"Adaptive Data Collection For High Fidelity Gaussian Splatting\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team4\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/","og_locale":"en_US","og_type":"article","og_title":"Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting","og_description":"Augment training images using diffusion models While 360\u00b0 video capture provides dense multi-view imagery as the photographer moves through the scene, this walking-based capture results in input images being constrained to a nearly constant elevation. Although this is sufficient for training views along the capture path, geometric consistency, particularly for tall structures and distant objects, &hellip; Continue reading \"Future works\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/","og_site_name":"Adaptive Data Collection For High Fidelity Gaussian Splatting","article_modified_time":"2025-12-13T00:00:02+00:00","og_image":[{"width":1388,"height":448,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/","name":"Future works - Adaptive Data Collection For High Fidelity Gaussian Splatting","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4-1024x331.png","datePublished":"2025-05-08T20:19:21+00:00","dateModified":"2025-12-13T00:00:02+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-content\/uploads\/sites\/120\/2025\/12\/image-4.png","width":1388,"height":448},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/next-steps\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/"},{"@type":"ListItem","position":2,"name":"Future works"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/","name":"Adaptive Data Collection For High Fidelity Gaussian Splatting","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/pages\/37","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/users\/233"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/comments?post=37"}],"version-history":[{"count":7,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/pages\/37\/revisions"}],"predecessor-version":[{"id":251,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/pages\/37\/revisions\/251"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team4\/wp-json\/wp\/v2\/media?parent=37"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}