{"id":23,"date":"2026-04-25T14:58:22","date_gmt":"2026-04-25T14:58:22","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/?page_id=23"},"modified":"2026-04-25T14:58:34","modified_gmt":"2026-04-25T14:58:34","slug":"method","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/","title":{"rendered":"Method"},"content":{"rendered":"\n

We propose a grounded vision-language pipeline that converts raw warehouse images into structured pallet attributes and an image-level safety decision. The overall flow is: pixels \u2192 objects \u2192 semantics \u2192 decisions<\/strong>. First, traditional computer vision localizes candidate pallet regions. Then, a vision-language model reasons over cropped pallet instances to extract semantic attributes such as load presence and damage. Finally, instance-level predictions are aggregated into a scene-level anomaly decision.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

We first use Grounding DINO to detect and localize candidate pallet regions in each warehouse image. This step is necessary because raw warehouse scenes are often cluttered, wide-angle, and contain multiple pallets at different scales. The detector outputs bounding boxes around likely pallet instances. Low-confidence detections are filtered out, duplicate or overlapping boxes are removed, and bounding boxes are slightly expanded before cropping to preserve visual context.<\/p>\n\n\n

\n
\"\"
Grounding DINO localizing pallet\/load regions in scene<\/figcaption><\/figure>\n<\/div>\n\n\n

Every crop that is detected is then entered into a vision-language model, which helps derive pallet features in a structured format. The vision-language model predicts the presence of the pallet and the presence of the load, identifies the type of the pallet, the presence of damage on the pallet, and the presence of damage on the load. Rather than using a single prompt for all the attributes, we separately query each attribute.<\/p>\n\n\n

\n
\"\"
Example VLM output given our set of prompts<\/figcaption><\/figure>\n<\/div>\n\n\n

Since a single warehouse image may consist of several pallets, individual predictions are made and aggregated to an overall image prediction. An anomaly is present in an image if at least one of the pallets or loads is predicted to be damaged. Such an approach is adopted due to the safety aspect of this task — one damaged pallet makes the situation hazardous.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

We propose a grounded vision-language pipeline that converts raw warehouse images into structured pallet attributes and an image-level safety decision. The overall flow is: pixels \u2192 objects \u2192 semantics \u2192 decisions. First, traditional computer vision localizes candidate pallet regions. Then, a vision-language model reasons over cropped pallet instances to extract semantic attributes such as load … <\/p>\n

Continue reading “Method”<\/span><\/a><\/p>\n","protected":false},"author":273,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-23","page","type-page","status-publish","hentry"],"yoast_head":"\nMethod - Grounded Vision-Language Understanding for Warehouse Automation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Method - Grounded Vision-Language Understanding for Warehouse Automation\" \/>\n<meta property=\"og:description\" content=\"We propose a grounded vision-language pipeline that converts raw warehouse images into structured pallet attributes and an image-level safety decision. The overall flow is: pixels \u2192 objects \u2192 semantics \u2192 decisions. First, traditional computer vision localizes candidate pallet regions. Then, a vision-language model reasons over cropped pallet instances to extract semantic attributes such as load … Continue reading "Method"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/\" \/>\n<meta property=\"og:site_name\" content=\"Grounded Vision-Language Understanding for Warehouse Automation\" \/>\n<meta property=\"article:modified_time\" content=\"2026-04-25T14:58:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-content\/uploads\/sites\/147\/2026\/04\/image-2.png\" \/>\n\t<meta property=\"og:image:width\" content=\"922\" \/>\n\t<meta property=\"og:image:height\" content=\"349\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/\",\"name\":\"Method - Grounded Vision-Language Understanding for Warehouse Automation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/wp-content\\\/uploads\\\/sites\\\/147\\\/2026\\\/04\\\/image-2.png\",\"datePublished\":\"2026-04-25T14:58:22+00:00\",\"dateModified\":\"2026-04-25T14:58:34+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/wp-content\\\/uploads\\\/sites\\\/147\\\/2026\\\/04\\\/image-2.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/wp-content\\\/uploads\\\/sites\\\/147\\\/2026\\\/04\\\/image-2.png\",\"width\":922,\"height\":349},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/method\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Method\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/\",\"name\":\"Grounded Vision-Language Understanding for Warehouse Automation\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf5\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Method - Grounded Vision-Language Understanding for Warehouse Automation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/","og_locale":"en_US","og_type":"article","og_title":"Method - Grounded Vision-Language Understanding for Warehouse Automation","og_description":"We propose a grounded vision-language pipeline that converts raw warehouse images into structured pallet attributes and an image-level safety decision. The overall flow is: pixels \u2192 objects \u2192 semantics \u2192 decisions. First, traditional computer vision localizes candidate pallet regions. Then, a vision-language model reasons over cropped pallet instances to extract semantic attributes such as load … Continue reading \"Method\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/","og_site_name":"Grounded Vision-Language Understanding for Warehouse Automation","article_modified_time":"2026-04-25T14:58:34+00:00","og_image":[{"width":922,"height":349,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-content\/uploads\/sites\/147\/2026\/04\/image-2.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/","name":"Method - Grounded Vision-Language Understanding for Warehouse Automation","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-content\/uploads\/sites\/147\/2026\/04\/image-2.png","datePublished":"2026-04-25T14:58:22+00:00","dateModified":"2026-04-25T14:58:34+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-content\/uploads\/sites\/147\/2026\/04\/image-2.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-content\/uploads\/sites\/147\/2026\/04\/image-2.png","width":922,"height":349},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/method\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/"},{"@type":"ListItem","position":2,"name":"Method"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/","name":"Grounded Vision-Language Understanding for Warehouse Automation","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/pages\/23","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/users\/273"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/comments?post=23"}],"version-history":[{"count":2,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/pages\/23\/revisions"}],"predecessor-version":[{"id":28,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/pages\/23\/revisions\/28"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf5\/wp-json\/wp\/v2\/media?parent=23"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}