{"id":39,"date":"2025-05-08T19:19:14","date_gmt":"2025-05-08T19:19:14","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/?page_id=39"},"modified":"2025-12-09T15:40:28","modified_gmt":"2025-12-09T15:40:28","slug":"overview","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","title":{"rendered":"Overview"},"content":{"rendered":"\n

Motivation<\/h2>\n\n\n\n

Medical Images, such as CT scans, MRIs, and X-rays, are commonly used in hospitals to help medical workers visualize internal structures of the human body. These images are crucial in helping the doctors understand information about the patient’s conditions. Medical image segmentation identifies organ boundaries, highlights important parts of the image, and isolates regions of interests such as the optic disc. This helps with accurate diagnosis, planning treatments, and monitoring diseases over time. Traditional ways require human labeling, which is expensive and time consuming. Most image segmentation models introduced recently require large datasets for training and pixel-level annotations [1] [2], requiring user expertise and prone to human error. In the general computer vision field, researchers have bridged language models with segmentation tasks [3]. This motivates us to explore text-based approach in segmentation within the medical field. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Image from MedSAM: “Segment Anything in Medical Images” by Ma et al., 2024<\/p>\n\n\n\n

Problem Statement<\/h2>\n\n\n\n

Traditionally, medical image segmentation relies on pixel-level annotations and training based on massive medical images. Recent approaches in the general computer vision field have explored reasoning-based segmentation [3], given a text prompt provided by the user. With this, we identify the need for a training-free, annotation-free medical image segmentation approach<\/strong> that minimizes manual effort while maintaining strong performance. The problem is how to accurately segment medical images based solely on flexible text prompts, without further task-specific training. We thus incorporate existing VLMs to generate pixel-level prompts from text prompts. Additionally, existing models like SAM [4] struggle with low-contrast medical images due to their training on diverse data, and medical segmentation models like MedSAM [1] require specific input prompts. To address this, we utilize a contrast enhancement method that enables us to use flexible prompts with SAM to achieve compatible performance with segmentation models specialized for medical images. <\/p>\n\n\n\n

Citations<\/h2>\n\n\n\n

[1] J. Ma, Y. He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nature Communications<\/em>, vol. 15, no. 1, Jan. 2024. [Online]. Available: http:\/\/dx.doi.org\/10.1038\/s41467-024-44824-z<\/a><\/p>\n\n\n\n

[2] Wu, Junde, Jiayuan Zhu, Yueming Jin, and Min Xu. “One-Prompt to Segment All Medical Images.” arXiv preprint arXiv:2305.10300<\/em> (2024). Available at: https:\/\/arxiv.org\/abs\/2305.10300<\/a><\/p>\n\n\n\n

[3] Wang, J., & Ke, L. (2024). LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning<\/em>. arXiv preprint arXiv:2404.08767<\/a><\/p>\n\n\n\n

[4] Kirillov, Alexander, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Doll\u00e1r, and Ross Girshick. “Segment Anything.” arXiv preprint arXiv:2304.02643<\/em> (2023). Available at: https:\/\/arxiv.org\/abs\/2304.02643<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Motivation Medical Images, such as CT scans, MRIs, and X-rays, are commonly used in hospitals to help medical workers visualize internal structures of the human body. These images are crucial in helping the doctors understand information about the patient’s conditions. Medical image segmentation identifies organ boundaries, highlights important parts of the image, and isolates regions … <\/p>\n

Continue reading “Overview”<\/span><\/a><\/p>\n","protected":false},"author":246,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-39","page","type-page","status-publish","hentry"],"yoast_head":"\nOverview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Overview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline\" \/>\n<meta property=\"og:description\" content=\"Motivation Medical Images, such as CT scans, MRIs, and X-rays, are commonly used in hospitals to help medical workers visualize internal structures of the human body. These images are crucial in helping the doctors understand information about the patient’s conditions. Medical image segmentation identifies organ boundaries, highlights important parts of the image, and isolates regions … Continue reading "Overview"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/\" \/>\n<meta property=\"og:site_name\" content=\"Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-09T15:40:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-content\/uploads\/sites\/130\/2025\/05\/Screenshot-2025-05-08-174832.png\" \/>\n\t<meta property=\"og:image:width\" content=\"668\" \/>\n\t<meta property=\"og:image:height\" content=\"494\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/\",\"name\":\"Overview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/wp-content\\\/uploads\\\/sites\\\/130\\\/2025\\\/05\\\/Screenshot-2025-05-08-174832.png\",\"datePublished\":\"2025-05-08T19:19:14+00:00\",\"dateModified\":\"2025-12-09T15:40:28+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/wp-content\\\/uploads\\\/sites\\\/130\\\/2025\\\/05\\\/Screenshot-2025-05-08-174832.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/wp-content\\\/uploads\\\/sites\\\/130\\\/2025\\\/05\\\/Screenshot-2025-05-08-174832.png\",\"width\":668,\"height\":494},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/\",\"name\":\"Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline\",\"description\":\"Student: Colleen Que | Collaborator: Qifeng Wu | Research Mentor: Xingjian Li | Advisor: Min Xu (CMU)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2025team7-1\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Overview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","og_locale":"en_US","og_type":"article","og_title":"Overview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline","og_description":"Motivation Medical Images, such as CT scans, MRIs, and X-rays, are commonly used in hospitals to help medical workers visualize internal structures of the human body. These images are crucial in helping the doctors understand information about the patient’s conditions. Medical image segmentation identifies organ boundaries, highlights important parts of the image, and isolates regions … Continue reading \"Overview\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","og_site_name":"Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline","article_modified_time":"2025-12-09T15:40:28+00:00","og_image":[{"width":668,"height":494,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-content\/uploads\/sites\/130\/2025\/05\/Screenshot-2025-05-08-174832.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","name":"Overview - Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-content\/uploads\/sites\/130\/2025\/05\/Screenshot-2025-05-08-174832.png","datePublished":"2025-05-08T19:19:14+00:00","dateModified":"2025-12-09T15:40:28+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-content\/uploads\/sites\/130\/2025\/05\/Screenshot-2025-05-08-174832.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-content\/uploads\/sites\/130\/2025\/05\/Screenshot-2025-05-08-174832.png","width":668,"height":494},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/"},{"@type":"ListItem","position":2,"name":"Overview"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/","name":"Medical Segmentation with Foundation Models: A Prompt-Based, Text-Guided, Training-Free Pipeline","description":"Student: Colleen Que | Collaborator: Qifeng Wu | Research Mentor: Xingjian Li | Advisor: Min Xu (CMU)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/pages\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/users\/246"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":10,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/pages\/39\/revisions"}],"predecessor-version":[{"id":113,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/pages\/39\/revisions\/113"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2025team7-1\/wp-json\/wp\/v2\/media?parent=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}