{"id":97,"date":"2026-05-05T20:52:45","date_gmt":"2026-05-05T20:52:45","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/?page_id=97"},"modified":"2026-05-07T15:53:24","modified_gmt":"2026-05-07T15:53:24","slug":"related-works","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/","title":{"rendered":"Related Works"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><strong>RAP &#8211; 3D Rasterization Augmented End-to-End Planning<\/strong><\/h2>\n\n\n\n<p>This work challenges the need for high-quality RGB rendering by proposing that planning task depends on geometry and dynamics rather than textures or lightning, enabling easier data augmentations. <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cross-agent synthesis<\/strong>: Generates diverse realistic views and interactions from other agents perspective.<\/li>\n\n\n\n<li><strong>Recovery-oriented perturbations<\/strong>: Increases model robustness by simulating recovery from distribution shifts. <\/li>\n\n\n\n<li><strong>Latent space alignment<\/strong>: Uses MSE loss (spatial-level alignment) and BCE loss (global-level alignment) to enforce feature consistency between the rasterized and real inputs. <\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"423\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-1024x423.png\" alt=\"\" class=\"wp-image-105\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-1024x423.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-300x124.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-768x317.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-1536x635.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13.png 1960w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?<\/strong><\/h2>\n\n\n\n<p>Existing datasets do not fully capture the complexity and variability of real world driving. InterPlan addresses this by introducing a benchmark to evaluate planning approaches on realistic long-tail scenarios. This work also proposes a hybrid planner combining LLM based behavioural planning with rule based motion planning. <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Augmented scenarios include jaywalking pedestrians, accident scenarios, overtaking maneuvers and construction scenarios.<\/li>\n\n\n\n<li>LLM based behavioural planner provides high level decision making with semantic understanding while rule based motion planner is used for precise trajectory generation. <\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"413\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14-1024x413.png\" alt=\"\" class=\"wp-image-114\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14-1024x413.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14-300x121.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14-768x309.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14-1536x619.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-14.png 1832w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios<\/strong><\/h2>\n\n\n\n<p>Proposes using natural language prompts to guide Vision-Language-Action models in handling complex, long-tail driving scenarios incorporating high-level semantic reasoning from VLMs into the lower level control loop. <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Automatic labelling pipeline uses LLMs to generate reasoning traces from driving data creating chain-of-thought explanations<\/li>\n\n\n\n<li>VLM provides high level semantic reasoning about the scene and VLA model execeytes precise control commands based on combined visual and semantic input. <\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"439\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15-1024x439.png\" alt=\"\" class=\"wp-image-116\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15-1024x439.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15-300x129.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15-768x329.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15-1536x658.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-15.png 1694w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/figure>\n\n\n\n<p><strong>References<\/strong><\/p>\n\n\n\n<p>[1] Feng, Lan, et al. &#8220;Rap: 3d rasterization augmented end-to-end planning.&#8221;\u00a0<em>arXiv preprint arXiv:2510.04333<\/em>\u00a0(2025).<\/p>\n\n\n\n<p>[2] Hallgarten, Marcel, et al. &#8220;Can vehicle motion planning generalize to realistic long-tail scenarios?.&#8221;\u00a0<em>2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)<\/em>. IEEE, 2024.<\/p>\n\n\n\n<p>[3] Gao, Tian, et al. &#8220;SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios.&#8221;\u00a0<em>arXiv preprint arXiv:2602.08440<\/em>\u00a0(2026).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>RAP &#8211; 3D Rasterization Augmented End-to-End Planning This work challenges the need for high-quality RGB rendering by proposing that planning task depends on geometry and dynamics rather than textures or lightning, enabling easier data augmentations. Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios? Existing datasets do not fully capture the complexity and variability of &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Related Works&#8221;<\/span><\/a><\/p>\n","protected":false},"author":285,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-97","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Related Works - Rasterized Representations for Autonomous Driving<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Related Works - Rasterized Representations for Autonomous Driving\" \/>\n<meta property=\"og:description\" content=\"RAP &#8211; 3D Rasterization Augmented End-to-End Planning This work challenges the need for high-quality RGB rendering by proposing that planning task depends on geometry and dynamics rather than textures or lightning, enabling easier data augmentations. Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios? Existing datasets do not fully capture the complexity and variability of &hellip; Continue reading &quot;Related Works&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/\" \/>\n<meta property=\"og:site_name\" content=\"Rasterized Representations for Autonomous Driving\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-07T15:53:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1960\" \/>\n\t<meta property=\"og:image:height\" content=\"810\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/\",\"name\":\"Related Works - Rasterized Representations for Autonomous Driving\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/wp-content\\\/uploads\\\/sites\\\/152\\\/2026\\\/05\\\/image-13-1024x423.png\",\"datePublished\":\"2026-05-05T20:52:45+00:00\",\"dateModified\":\"2026-05-07T15:53:24+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/wp-content\\\/uploads\\\/sites\\\/152\\\/2026\\\/05\\\/image-13.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/wp-content\\\/uploads\\\/sites\\\/152\\\/2026\\\/05\\\/image-13.png\",\"width\":1960,\"height\":810},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/related-works\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Related Works\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/\",\"name\":\"Rasterized Representations for Autonomous Driving\",\"description\":\"Nikita Malik, Karthik Pullalarevu, Kris Kitani, Nicolas Ugrinovic, Varun Ramakrishna, Micol Marchetti-Bowick  \",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2026teamf10\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Related Works - Rasterized Representations for Autonomous Driving","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/","og_locale":"en_US","og_type":"article","og_title":"Related Works - Rasterized Representations for Autonomous Driving","og_description":"RAP &#8211; 3D Rasterization Augmented End-to-End Planning This work challenges the need for high-quality RGB rendering by proposing that planning task depends on geometry and dynamics rather than textures or lightning, enabling easier data augmentations. Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios? Existing datasets do not fully capture the complexity and variability of &hellip; Continue reading \"Related Works\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/","og_site_name":"Rasterized Representations for Autonomous Driving","article_modified_time":"2026-05-07T15:53:24+00:00","og_image":[{"width":1960,"height":810,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/","name":"Related Works - Rasterized Representations for Autonomous Driving","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13-1024x423.png","datePublished":"2026-05-05T20:52:45+00:00","dateModified":"2026-05-07T15:53:24+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-content\/uploads\/sites\/152\/2026\/05\/image-13.png","width":1960,"height":810},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/related-works\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/"},{"@type":"ListItem","position":2,"name":"Related Works"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/","name":"Rasterized Representations for Autonomous Driving","description":"Nikita Malik, Karthik Pullalarevu, Kris Kitani, Nicolas Ugrinovic, Varun Ramakrishna, Micol Marchetti-Bowick  ","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/pages\/97","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/users\/285"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/comments?post=97"}],"version-history":[{"count":9,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/pages\/97\/revisions"}],"predecessor-version":[{"id":119,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/pages\/97\/revisions\/119"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf10\/wp-json\/wp\/v2\/media?parent=97"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}