{"id":103,"date":"2025-12-13T01:35:47","date_gmt":"2025-12-13T01:35:47","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/?page_id=103"},"modified":"2025-12-13T02:09:27","modified_gmt":"2025-12-13T02:09:27","slug":"experiment","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/","title":{"rendered":"Experiments"},"content":{"rendered":"\n

Visualization of Temporally Hierarchical Scene Graph<\/h3>\n\n\n\n
\"\"
Visualization of Temporally Hierarchical Scene Graph<\/figcaption><\/figure>\n\n\n\n

VQA Quantitative Results<\/h3>\n\n\n\n
\"\"<\/figure>\n\n\n\n

The figure above presents the quantitative performance of different methods on the VQA task, measured across eight sub-tasks: Overall, 3D Perception, Fixture Location, Fine-Grained Action Localization, Fine-Grained Action Recognition, Fine-Grained How Recognition, Fine-Grained Why Recognition, Gaze Estimation, and Gaze Interaction Anticipation.<\/p>\n\n\n\n

We compare four different approaches:<\/p>\n\n\n\n

Blind Guess<\/strong>, which relies solely on the question without any scene information, achieves the lowest performance overall, with accuracy typically ranging from 0.1 to 0.3. This highlights the difficulty of answering questions correctly without access to visual context.<\/p>\n\n\n\n

Per-Frame Graph (ours)<\/strong> leverages a full scene graph for each frame. It demonstrates a significant improvement over Blind Guess, particularly in Gaze Estimation and Interaction Anticipation, indicating that frame-level graph information effectively captures actions and gaze relationships.<\/p>\n\n\n\n

THSG (ours)<\/strong> incorporates a hierarchical pruning strategy. Compared to Per-Frame Graph, it shows further improvement in some tasks, such as Fine-Grained Action Localization and Overall performance, demonstrating the effectiveness of reducing redundant information.<\/p>\n\n\n\n

SceneNet<\/strong>[1], as a state-of-the-art baseline, performs well on 3D Perception and Fine-Grained How Recognition but slightly underperforms our Per-Frame Graph in Gaze Estimation and Interaction Anticipation.<\/p>\n\n\n\n

Overall, our Per-Frame Graph and THSG methods consistently outperform Blind Guess and SceneNet across most sub-tasks, with notable gains in action recognition, gaze prediction, and interaction anticipation. These results validate the effectiveness of incorporating scene graphs and hierarchical strategies in VQA tasks.<\/p>\n\n\n\n

References<\/h2>\n\n\n\n

[1] A. Taluzzi, et al. \u201cFrom Pixels to Graphs: using Scene and Knowledge Graphs for HD-EPIC VQA Challenge\u201d arXiv:2506.08553 (2025)<\/em>. https:\/\/arxiv.org\/abs\/2506.08553<\/a><\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Visualization of Temporally Hierarchical Scene Graph VQA Quantitative Results The figure above presents the quantitative performance of different methods on the VQA task, measured across eight sub-tasks: Overall, 3D Perception, Fixture Location, Fine-Grained Action Localization, Fine-Grained Action Recognition, Fine-Grained How Recognition, Fine-Grained Why Recognition, Gaze Estimation, and Gaze Interaction Anticipation. We compare four different approaches: … <\/p>\n

Continue reading “Experiments”<\/span><\/a><\/p>\n","protected":false},"author":239,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-103","page","type-page","status-publish","hentry"],"yoast_head":"\nExperiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Experiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b\" \/>\n<meta property=\"og:description\" content=\"Visualization of Temporally Hierarchical Scene Graph VQA Quantitative Results The figure above presents the quantitative performance of different methods on the VQA task, measured across eight sub-tasks: Overall, 3D Perception, Fixture Location, Fine-Grained Action Localization, Fine-Grained Action Recognition, Fine-Grained How Recognition, Fine-Grained Why Recognition, Gaze Estimation, and Gaze Interaction Anticipation. We compare four different approaches: … Continue reading "Experiments"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/\" \/>\n<meta property=\"og:site_name\" content=\"Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-13T02:09:27+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-content\/uploads\/sites\/124\/2025\/12\/visualization.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1282\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/\",\"name\":\"Experiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/wp-content\\\/uploads\\\/sites\\\/124\\\/2025\\\/12\\\/visualization-1024x479.png\",\"datePublished\":\"2025-12-13T01:35:47+00:00\",\"dateModified\":\"2025-12-13T02:09:27+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/wp-content\\\/uploads\\\/sites\\\/124\\\/2025\\\/12\\\/visualization.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/wp-content\\\/uploads\\\/sites\\\/124\\\/2025\\\/12\\\/visualization.png\",\"width\":1282,\"height\":600},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/experiment\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Experiments\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/\",\"name\":\"Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team17\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Experiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/","og_locale":"en_US","og_type":"article","og_title":"Experiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b","og_description":"Visualization of Temporally Hierarchical Scene Graph VQA Quantitative Results The figure above presents the quantitative performance of different methods on the VQA task, measured across eight sub-tasks: Overall, 3D Perception, Fixture Location, Fine-Grained Action Localization, Fine-Grained Action Recognition, Fine-Grained How Recognition, Fine-Grained Why Recognition, Gaze Estimation, and Gaze Interaction Anticipation. We compare four different approaches: … Continue reading \"Experiments\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/","og_site_name":"Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b","article_modified_time":"2025-12-13T02:09:27+00:00","og_image":[{"width":1282,"height":600,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-content\/uploads\/sites\/124\/2025\/12\/visualization.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/","name":"Experiments - Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-content\/uploads\/sites\/124\/2025\/12\/visualization-1024x479.png","datePublished":"2025-12-13T01:35:47+00:00","dateModified":"2025-12-13T02:09:27+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-content\/uploads\/sites\/124\/2025\/12\/visualization.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-content\/uploads\/sites\/124\/2025\/12\/visualization.png","width":1282,"height":600},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/experiment\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/"},{"@type":"ListItem","position":2,"name":"Experiments"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/","name":"Temporally Hierarchical Scene Graph Generation for Video Question Answering\u200b","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/pages\/103","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/users\/239"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/comments?post=103"}],"version-history":[{"count":4,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/pages\/103\/revisions"}],"predecessor-version":[{"id":113,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/pages\/103\/revisions\/113"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team17\/wp-json\/wp\/v2\/media?parent=103"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}