The DARPA Triage Challenge aims to enable autonomous systems to rapidly detect and prioritize casualties in disaster scenarios using UAV imagery. In this project, we develop a multimodal medical triaging pipeline that uses RGB and thermal (IR) drone data to predict 50+ hierarchical medical labels, including hemorrhage, trauma, posture, and respiratory distress.
A major challenge in this setting is the limited availability of paired RGB–IR training data. To address this, we first train a conditional diffusion model to generate thermal images from RGB inputs, enabling large-scale synthetic RGB–IR pair generation. These multimodal pairs are then used to fine-tune Vision-Language Models (VLMs) for robust casualty understanding and triage reasoning in complex real-world environments.