{"id":118,"date":"2025-12-13T01:43:10","date_gmt":"2025-12-13T01:43:10","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/?page_id=118"},"modified":"2025-12-13T02:00:36","modified_gmt":"2025-12-13T02:00:36","slug":"results-2","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/","title":{"rendered":"Results"},"content":{"rendered":"\n

Quantitative Results<\/h2>\n\n\n\n

Ablation on approach. <\/strong>We perform an ablation study to evaluate the impact of both the optimization strategy and camera parameterization on performance. We compare random sampling<\/strong>, CMA-ES<\/strong>, and our gradient-based ascent method<\/strong> across all tested rotation representations.<\/p>\n\n\n\n

Random sampling provides an unbiased baseline for probing sensitivity to camera perturbations, while CMA-ES improves search efficiency through adaptive covariance estimation. Our gradient-based approach consistently outperforms both baselines<\/strong>, converging faster and identifying more severe failure cases across all parameterizations.<\/p>\n\n\n\n

These results highlight the importance of jointly selecting an effective optimization method and a stable camera representation when evaluating depth model robustness.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Ablation on model. <\/strong>Table below reports the average performance of all evaluated models across scenes, summarizing their robustness under adversarial camera perturbations. We report both \u03b4<\/mi>1<\/mn><\/msub><\/mrow>\\delta_1<\/annotation><\/semantics><\/math>\u200b and AbsRel, where higher AbsRel and lower \u03b4<\/mi>1<\/mn><\/msub><\/mrow>\\delta_1<\/annotation><\/semantics><\/math>indicate greater discrepancy between predictions from adversarial viewpoints and ground truth depth. Overall, more recent models, particularly DepthAnything V2<\/strong>, exhibit stronger robustness, achieving higher \u03b4<\/mi>1<\/mn><\/msub><\/mrow>\\delta_1<\/annotation><\/semantics><\/math>\u200b and lower AbsRel on average compared to older architectures such as MiDaS<\/strong>. Models benefiting from large-scale synthetic pretraining or teacher-based supervision consistently outperform earlier methods, highlighting the impact of modern training strategies on depth stability under viewpoint shifts.<\/p>\n\n\n
\n
\"\"<\/figure>\n<\/div>\n\n\n

Qualitative Results<\/h2>\n\n\n\n

Below we show some failure modes discovered by our model.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Camera Trajectories<\/strong>. We also show some examples of the smoothness of trajectories under the R6 parameterization as opposed to other ones below.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Discussion<\/h1>\n\n\n\n

Stochasticity of Trajectories<\/h2>\n\n\n\n

One issue is resolving degenerate solutions and trajectory stability. Specifically, we’ve found that trajectories initialized from the same starting point do not resolve well even by seeding PyTorch3D. We suspect that it’s the result of two issues.<\/p>\n\n\n\n

    \n
  1. Camera Clipping:<\/strong> We observe that degenerate solutions<\/strong>, such as camera clipping through geometry, often lead to large spikes in error<\/strong> and hinder convergence. These artifacts introduce instability and result in suboptimal optimization trajectories<\/strong>, particularly when the camera gets too close to or intersects with scene surfaces.<\/li>\n\n\n\n
  2. Loss Surface Instability:<\/strong> Additionally, we suspect that the instability of the optimization process<\/strong> is exacerbated by the fact that the ground truth signal changes at every frame, due to the adversarial nature of the supervision<\/strong>. This leads to a non-stationary loss surface, making it harder for the optimizer to converge consistently. To address this, and inspired by prior work in PyTorch3D, we plan to explore the use of Soft Rasterization<\/strong> to produce smoother gradients<\/strong> and reduce the volatility of the loss surface. This may help guide the optimization more reliably, especially in challenging regions of the parameter space.<\/li>\n<\/ol>\n\n\n\n
    \"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"

    Quantitative Results Ablation on approach. We perform an ablation study to evaluate the impact of both the optimization strategy and camera parameterization on performance. We compare random sampling, CMA-ES, and our gradient-based ascent method across all tested rotation representations. Random sampling provides an unbiased baseline for probing sensitivity to camera perturbations, while CMA-ES improves search … <\/p>\n

    Continue reading “Results”<\/span><\/a><\/p>\n","protected":false},"author":240,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-118","page","type-page","status-publish","hentry"],"yoast_head":"\nResults - Breaking Depth Estimation Models with Semantic Adversarial Attacks<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Results - Breaking Depth Estimation Models with Semantic Adversarial Attacks\" \/>\n<meta property=\"og:description\" content=\"Quantitative Results Ablation on approach. We perform an ablation study to evaluate the impact of both the optimization strategy and camera parameterization on performance. We compare random sampling, CMA-ES, and our gradient-based ascent method across all tested rotation representations. Random sampling provides an unbiased baseline for probing sensitivity to camera perturbations, while CMA-ES improves search … Continue reading "Results"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Breaking Depth Estimation Models with Semantic Adversarial Attacks\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-13T02:00:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-content\/uploads\/sites\/125\/2025\/12\/image-3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"432\" \/>\n\t<meta property=\"og:image:height\" content=\"180\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/\",\"name\":\"Results - Breaking Depth Estimation Models with Semantic Adversarial Attacks\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/wp-content\\\/uploads\\\/sites\\\/125\\\/2025\\\/12\\\/image-3.png\",\"datePublished\":\"2025-12-13T01:43:10+00:00\",\"dateModified\":\"2025-12-13T02:00:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/wp-content\\\/uploads\\\/sites\\\/125\\\/2025\\\/12\\\/image-3.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/wp-content\\\/uploads\\\/sites\\\/125\\\/2025\\\/12\\\/image-3.png\",\"width\":432,\"height\":180},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/results-2\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Results\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/\",\"name\":\"Breaking Depth Estimation Models with Semantic Adversarial Attacks\",\"description\":\"Adithya Narayan, Utkarsh Ohja, Justin Theiss, Aayush Prakash, Fernando De la Torre\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team12-2\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Results - Breaking Depth Estimation Models with Semantic Adversarial Attacks","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/","og_locale":"en_US","og_type":"article","og_title":"Results - Breaking Depth Estimation Models with Semantic Adversarial Attacks","og_description":"Quantitative Results Ablation on approach. We perform an ablation study to evaluate the impact of both the optimization strategy and camera parameterization on performance. We compare random sampling, CMA-ES, and our gradient-based ascent method across all tested rotation representations. Random sampling provides an unbiased baseline for probing sensitivity to camera perturbations, while CMA-ES improves search … Continue reading \"Results\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/","og_site_name":"Breaking Depth Estimation Models with Semantic Adversarial Attacks","article_modified_time":"2025-12-13T02:00:36+00:00","og_image":[{"width":432,"height":180,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-content\/uploads\/sites\/125\/2025\/12\/image-3.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/","name":"Results - Breaking Depth Estimation Models with Semantic Adversarial Attacks","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-content\/uploads\/sites\/125\/2025\/12\/image-3.png","datePublished":"2025-12-13T01:43:10+00:00","dateModified":"2025-12-13T02:00:36+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-content\/uploads\/sites\/125\/2025\/12\/image-3.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-content\/uploads\/sites\/125\/2025\/12\/image-3.png","width":432,"height":180},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/results-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/"},{"@type":"ListItem","position":2,"name":"Results"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/","name":"Breaking Depth Estimation Models with Semantic Adversarial Attacks","description":"Adithya Narayan, Utkarsh Ohja, Justin Theiss, Aayush Prakash, Fernando De la Torre","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/pages\/118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/users\/240"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/comments?post=118"}],"version-history":[{"count":4,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/pages\/118\/revisions"}],"predecessor-version":[{"id":133,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/pages\/118\/revisions\/133"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team12-2\/wp-json\/wp\/v2\/media?parent=118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}