{"id":25,"date":"2025-05-09T04:15:58","date_gmt":"2025-05-09T04:15:58","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/?page_id=25"},"modified":"2025-12-13T02:06:44","modified_gmt":"2025-12-13T02:06:44","slug":"methods","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/","title":{"rendered":"Methods"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">How Can Flow Supervise Geometry?<\/h2>\n\n\n\n<p>Modern visual-geometry networks first encode each image into patch tokens and a camera token&nbsp;<strong>(a)<\/strong>&nbsp;using a multi-view transformer backbone. Based on these latent features, there are several ways to predict dense correspondences between frames. Traditional correspondence heads&nbsp;<strong>(b)<\/strong>&nbsp;infer flow directly from patch features, relying purely on visual appearance and ignoring the underlying scene geometry. Alternatively, one may compute flow by explicitly projecting predicted 3D points into another view using decoded camera poses&nbsp;<strong>(c)<\/strong>; however, this approach assumes static scenes and is highly sensitive to geometric prediction errors. In contrast, our factored flow mechanism&nbsp;<strong>(d)<\/strong>&nbsp;combines the geometry latents from the source view with the camera latents from the target view and decodes correspondences directly in latent space. This design yields geometry-aware flow, improves robustness, and naturally extends to dynamic scenes.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"290\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1024x290.png\" alt=\"\" class=\"wp-image-43\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1024x290.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-300x85.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-768x218.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1536x436.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image.png 1918w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p>Here we propose Flow3r, which predicts visual geometry using factored flow supervision, enabling scalable geometry learning from unlabeled videos. Each input image is encoded and processed by the multi-view transformer to produce camera tokens and patch tokens. For data with dense geometry and pose labels, we directly supervise the patch tokens and camera tokens with the corresponding labels. For unlabeled datasets without geometry and supervision, we predict flow between two frames in a factorized manner, supervised by an off-the-shelf 2D flow prediction model. To obtain the factored flow, we fuse the patch features of one frame with the camera features of the other, and decode the fused representation through the DPT head to produce dense flow predictions.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"399\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1-1024x399.png\" alt=\"\" class=\"wp-image-45\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1-1024x399.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1-300x117.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1-768x300.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1-1536x599.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1.png 1902w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>How Can Flow Supervise Geometry? Modern visual-geometry networks first encode each image into patch tokens and a camera token&nbsp;(a)&nbsp;using a multi-view transformer backbone. Based on these latent features, there are several ways to predict dense correspondences between frames. Traditional correspondence heads&nbsp;(b)&nbsp;infer flow directly from patch features, relying purely on visual appearance and ignoring the underlying &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Methods&#8221;<\/span><\/a><\/p>\n","protected":false},"author":242,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-25","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning\" \/>\n<meta property=\"og:description\" content=\"How Can Flow Supervise Geometry? Modern visual-geometry networks first encode each image into patch tokens and a camera token&nbsp;(a)&nbsp;using a multi-view transformer backbone. Based on these latent features, there are several ways to predict dense correspondences between frames. Traditional correspondence heads&nbsp;(b)&nbsp;infer flow directly from patch features, relying purely on visual appearance and ignoring the underlying &hellip; Continue reading &quot;Methods&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/\" \/>\n<meta property=\"og:site_name\" content=\"Flow3r: Factored Flow Prediction for Visual Geometry Learning\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-13T02:06:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1918\" \/>\n\t<meta property=\"og:image:height\" content=\"544\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/\",\"name\":\"Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/wp-content\\\/uploads\\\/sites\\\/127\\\/2025\\\/12\\\/image-1024x290.png\",\"datePublished\":\"2025-05-09T04:15:58+00:00\",\"dateModified\":\"2025-12-13T02:06:44+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/wp-content\\\/uploads\\\/sites\\\/127\\\/2025\\\/12\\\/image.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/wp-content\\\/uploads\\\/sites\\\/127\\\/2025\\\/12\\\/image.png\",\"width\":1918,\"height\":544},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/methods\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Methods\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/\",\"name\":\"Flow3r: Factored Flow Prediction for Visual Geometry Learning\",\"description\":\"Author: Clara Cong; Advisor: Shubham Tulsiani\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team8-1\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/","og_locale":"en_US","og_type":"article","og_title":"Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning","og_description":"How Can Flow Supervise Geometry? Modern visual-geometry networks first encode each image into patch tokens and a camera token&nbsp;(a)&nbsp;using a multi-view transformer backbone. Based on these latent features, there are several ways to predict dense correspondences between frames. Traditional correspondence heads&nbsp;(b)&nbsp;infer flow directly from patch features, relying purely on visual appearance and ignoring the underlying &hellip; Continue reading \"Methods\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/","og_site_name":"Flow3r: Factored Flow Prediction for Visual Geometry Learning","article_modified_time":"2025-12-13T02:06:44+00:00","og_image":[{"width":1918,"height":544,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/","name":"Methods - Flow3r: Factored Flow Prediction for Visual Geometry Learning","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image-1024x290.png","datePublished":"2025-05-09T04:15:58+00:00","dateModified":"2025-12-13T02:06:44+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-content\/uploads\/sites\/127\/2025\/12\/image.png","width":1918,"height":544},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/methods\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/"},{"@type":"ListItem","position":2,"name":"Methods"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/","name":"Flow3r: Factored Flow Prediction for Visual Geometry Learning","description":"Author: Clara Cong; Advisor: Shubham Tulsiani","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/pages\/25","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/users\/242"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/comments?post=25"}],"version-history":[{"count":5,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/pages\/25\/revisions"}],"predecessor-version":[{"id":64,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/pages\/25\/revisions\/64"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team8-1\/wp-json\/wp\/v2\/media?parent=25"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}