{"id":33,"date":"2025-05-06T04:21:52","date_gmt":"2025-05-06T04:21:52","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/?page_id=33"},"modified":"2025-05-06T16:10:37","modified_gmt":"2025-05-06T16:10:37","slug":"multi-image-hallucination","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/","title":{"rendered":"Multi-image Hallucination"},"content":{"rendered":"\n<p>Hallucination means LLM or LVLM generates content that is inconsistent with fact, where facts come from provided images, language prompts, or existing knowledge bases. For instance, if we let an LVLM like InternVL-2.5-8B describe this jersey in detail, it gives a very accurate response, including color, number, and text.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\" alt=\"\" style=\"object-fit:cover\" \/><figcaption class=\"wp-element-caption\">Figure 1: Single-image VQA of InternVL-2.5-8B. Text Prompt: &#8220;Describe this image in detail&#8221;. Response: The shirt is a <strong>purple<\/strong> basketball jersey with the <strong>number 23<\/strong> prominently displayed in yellow with a purple outline on the front. The word &#8220;<strong>LAKESHOW<\/strong>&#8221; is printed above the number in yellow capital letters.<\/figcaption><\/figure>\n\n\n\n<p>However, suppose we put this image in an image sequence and ask the model to choose which images show the same shirt in the input images, although InternVL-2.5 can correctly describe the first and third one, it wrongly recognizes the 2nd jersey as white color with text \u201cGolden State\u201d, thus incorrectly classifying the 2nd and 3rd shirts as the same category. Therefore, in the context of multi-image tasks emphasizing perceiving and linking visual cues, the wrong perception of details when comparing across multiple images indicates hallucination.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUcaPmPTLrJVqBhyUry2QYQAwqQwMH7Ckr65-7i4Css6KXL8OaSmyrHXScka0XfRF-C9KFEJZkU8l4x0i_Af68O6YQqa_rIG03P0LV1eyvNHBr7pGHV2WApS9YIUiQ4VdJXnq8hfwQ=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\" alt=\"\" \/><figcaption class=\"wp-element-caption\">Figure 2: Text prompt:\u00a0Which images show the same shirt in the input images? Choices: A) 1 and 2; B) 2 and 3; C) None; D) 1 and 3. Response: To determine which images show the same shirt, I need to compare the jerseys in the images. The first image shows a <strong>purple &#8220;LAKE SHOW&#8221;<\/strong> jersey with the <strong>number 23<\/strong>. The <strong>second image<\/strong> shows a <strong>white &#8220;Golden State&#8221;<\/strong> jersey with the <strong>number 23<\/strong>. The third image shows a <strong>white &#8220;Golden State&#8221;<\/strong> jersey with the <strong>number 23<\/strong>. Since the second and third images both show the &#8220;Golden State&#8221; jersey, the correct answer is B) 2 and 3. MY ANSWER: <strong>B) 2 and 3<\/strong>.<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Hallucination means LLM or LVLM generates content that is inconsistent with fact, where facts come from provided images, language prompts, or existing knowledge bases. For instance, if we let an LVLM like InternVL-2.5-8B describe this jersey in detail, it gives a very accurate response, including color, number, and text. However, suppose we put this image &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Multi-image Hallucination&#8221;<\/span><\/a><\/p>\n","protected":false},"author":230,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-33","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Multi-image Hallucination - LVLM Multi-image VQA Dehallucination<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Multi-image Hallucination - LVLM Multi-image VQA Dehallucination\" \/>\n<meta property=\"og:description\" content=\"Hallucination means LLM or LVLM generates content that is inconsistent with fact, where facts come from provided images, language prompts, or existing knowledge bases. For instance, if we let an LVLM like InternVL-2.5-8B describe this jersey in detail, it gives a very accurate response, including color, number, and text. However, suppose we put this image &hellip; Continue reading &quot;Multi-image Hallucination&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/\" \/>\n<meta property=\"og:site_name\" content=\"LVLM Multi-image VQA Dehallucination\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-06T16:10:37+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/\",\"name\":\"Multi-image Hallucination - LVLM Multi-image VQA Dehallucination\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/lh7-rt.googleusercontent.com\\\/slidesz\\\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\",\"datePublished\":\"2025-05-06T04:21:52+00:00\",\"dateModified\":\"2025-05-06T16:10:37+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/#primaryimage\",\"url\":\"https:\\\/\\\/lh7-rt.googleusercontent.com\\\/slidesz\\\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\",\"contentUrl\":\"https:\\\/\\\/lh7-rt.googleusercontent.com\\\/slidesz\\\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/multi-image-hallucination\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Multi-image Hallucination\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/\",\"name\":\"LVLM Multi-image VQA Dehallucination\",\"description\":\"Authors: Dongyu Yao, Keling Yao, Harry Chi; Advisor: Prof. Katia Sycara and Dr. Yaqi Xie\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team15\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Multi-image Hallucination - LVLM Multi-image VQA Dehallucination","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/","og_locale":"en_US","og_type":"article","og_title":"Multi-image Hallucination - LVLM Multi-image VQA Dehallucination","og_description":"Hallucination means LLM or LVLM generates content that is inconsistent with fact, where facts come from provided images, language prompts, or existing knowledge bases. For instance, if we let an LVLM like InternVL-2.5-8B describe this jersey in detail, it gives a very accurate response, including color, number, and text. However, suppose we put this image &hellip; Continue reading \"Multi-image Hallucination\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/","og_site_name":"LVLM Multi-image VQA Dehallucination","article_modified_time":"2025-05-06T16:10:37+00:00","og_image":[{"url":"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/","name":"Multi-image Hallucination - LVLM Multi-image VQA Dehallucination","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/#primaryimage"},"thumbnailUrl":"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h","datePublished":"2025-05-06T04:21:52+00:00","dateModified":"2025-05-06T16:10:37+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/#primaryimage","url":"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h","contentUrl":"https:\/\/lh7-rt.googleusercontent.com\/slidesz\/AGV_vUeJTOwvlD_vuLo6z_e28aAsDr5_NQ6vW7qYC-F0-0T7NbD77XYMCS-A0zSUm4TgEKpYtznRhXyi-pBDWJmnv-z85V0d9Bv0Qp--f2P1umJqu-wlyOM02p11Y3yuF7XzbhKtfD8OOg=s2048?key=MhxsOoUwM9UCIcXKZDmfhW2h"},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/multi-image-hallucination\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/"},{"@type":"ListItem","position":2,"name":"Multi-image Hallucination"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/","name":"LVLM Multi-image VQA Dehallucination","description":"Authors: Dongyu Yao, Keling Yao, Harry Chi; Advisor: Prof. Katia Sycara and Dr. Yaqi Xie","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/pages\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/users\/230"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/comments?post=33"}],"version-history":[{"count":3,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/pages\/33\/revisions"}],"predecessor-version":[{"id":51,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/pages\/33\/revisions\/51"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team15\/wp-json\/wp\/v2\/media?parent=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}