{"id":22,"date":"2021-12-10T04:21:15","date_gmt":"2021-12-10T04:21:15","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/?page_id=22"},"modified":"2022-05-01T20:01:16","modified_gmt":"2022-05-02T01:01:16","slug":"reinforcement-learning-for-self-driving-cars","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","title":{"rendered":""},"content":{"rendered":"\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<h1 class=\"wp-block-heading\"><strong>Project Summary<\/strong><\/h1>\n\n\n\n<p class=\"has-cyan-bluish-gray-background-color has-background\">Improve Reinforcement Learning (RL) based self-driving algorithms by adding semantic-segmented bird&#8217;s-eye-view (BEV) images to the state space. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"128\" height=\"128\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png\" alt=\"\" class=\"wp-image-93\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png 128w, https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3-100x100.png 100w\" sizes=\"auto, (max-width: 128px) 100vw, 128px\" \/><figcaption>semantic segmented bird&#8217;s eye view images generated from Carla<\/figcaption><\/figure><\/div>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"126\" height=\"128\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/2.png\" alt=\"\" class=\"wp-image-92\" \/><figcaption>Each color denotes a distinct class of objects<\/figcaption><\/figure><\/div>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"127\" height=\"128\" src=\"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/1.png\" alt=\"\" class=\"wp-image-91\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/1.png 127w, https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/1-100x100.png 100w\" sizes=\"auto, (max-width: 127px) 100vw, 127px\" \/><figcaption>The red vehicle at the centre is the ego vehicle<\/figcaption><\/figure><\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<h2 class=\"wp-block-heading\"><strong>Motivation<\/strong><\/h2>\n\n\n\n<p>Current RL algorithms used to learn autonomous navigation at <a href=\"https:\/\/www.autonlab.org\/\">Auton Lab<\/a> uses <font color=\"#222222\"><a href=\"https:\/\/carla.org\/\">Carla <\/a>simu<\/font>lator as the source of training and testing data. The algorithms utilize the Ground truth information like distance to nearest vehicle, state of traffic lights and deviation from the planned trajectory among others to learn how to drive. Such information is hard to come by in the real world. <\/p>\n\n\n\n<p>We hope to bring in more visual information from which such state cues can be inferred rather than measured, and hence bring autonomous driving closer to the real world.  <\/p>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Problem<\/strong><\/h2>\n\n\n\n<p>We plan to integrate semantic segmented images from a bird&#8217;s eye view camera position into the state space as an alternative to the GT sensor state space currently used. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Solution<\/strong><\/h2>\n\n\n\n<p>We use a multi-headed current frame stack encoder and future frame stack predictor to create a latent embedding of a temporal sequence which will help the RL model make better decisions.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n<\/div>\n<\/div>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Project Summary Improve Reinforcement Learning (RL) based self-driving algorithms by adding semantic-segmented bird&#8217;s-eye-view (BEV) images to the state space. Motivation Current RL algorithms used to learn autonomous navigation at Auton Lab uses Carla simulator as the source of training and testing data. The algorithms utilize the Ground truth information like distance to nearest vehicle, state &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;&#8221;<\/span><\/a><\/p>\n","protected":false},"author":98,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-22","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>- Reinforcement Learning for Self Driving Cars<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"- Reinforcement Learning for Self Driving Cars\" \/>\n<meta property=\"og:description\" content=\"Project Summary Improve Reinforcement Learning (RL) based self-driving algorithms by adding semantic-segmented bird&#8217;s-eye-view (BEV) images to the state space. Motivation Current RL algorithms used to learn autonomous navigation at Auton Lab uses Carla simulator as the source of training and testing data. The algorithms utilize the Ground truth information like distance to nearest vehicle, state &hellip; Continue reading &quot;&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/\" \/>\n<meta property=\"og:site_name\" content=\"Reinforcement Learning for Self Driving Cars\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-02T01:01:16+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/\",\"name\":\"- Reinforcement Learning for Self Driving Cars\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/wp-content\\\/uploads\\\/sites\\\/46\\\/2022\\\/05\\\/3.png\",\"datePublished\":\"2021-12-10T04:21:15+00:00\",\"dateModified\":\"2022-05-02T01:01:16+00:00\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#primaryimage\",\"url\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/wp-content\\\/uploads\\\/sites\\\/46\\\/2022\\\/05\\\/3.png\",\"contentUrl\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/wp-content\\\/uploads\\\/sites\\\/46\\\/2022\\\/05\\\/3.png\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/\",\"name\":\"Reinforcement Learning for Self Driving Cars\",\"description\":\"Improving RL based self-driving using visual input | Students: Manikandtan Chiral Kunjunni Kartha, Hao-Ming Fu | Advisor: Jeff Schneider (CMU) | Sponsor: Argo AI\",\"publisher\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#organization\",\"name\":\"Reinforcement Learning for Self Driving Cars\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/wp-content\\\/uploads\\\/sites\\\/46\\\/2022\\\/04\\\/cropped-car-4343634-scaled-1.jpg\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/wp-content\\\/uploads\\\/sites\\\/46\\\/2022\\\/04\\\/cropped-car-4343634-scaled-1.jpg\",\"width\":288,\"height\":250,\"caption\":\"Reinforcement Learning for Self Driving Cars\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teama\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"- Reinforcement Learning for Self Driving Cars","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","og_locale":"en_US","og_type":"article","og_title":"- Reinforcement Learning for Self Driving Cars","og_description":"Project Summary Improve Reinforcement Learning (RL) based self-driving algorithms by adding semantic-segmented bird&#8217;s-eye-view (BEV) images to the state space. Motivation Current RL algorithms used to learn autonomous navigation at Auton Lab uses Carla simulator as the source of training and testing data. The algorithms utilize the Ground truth information like distance to nearest vehicle, state &hellip; Continue reading \"\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","og_site_name":"Reinforcement Learning for Self Driving Cars","article_modified_time":"2022-05-02T01:01:16+00:00","og_image":[{"url":"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","name":"- Reinforcement Learning for Self Driving Cars","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#primaryimage"},"thumbnailUrl":"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png","datePublished":"2021-12-10T04:21:15+00:00","dateModified":"2022-05-02T01:01:16+00:00","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#primaryimage","url":"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png","contentUrl":"http:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/05\/3.png"},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","name":"Reinforcement Learning for Self Driving Cars","description":"Improving RL based self-driving using visual input | Students: Manikandtan Chiral Kunjunni Kartha, Hao-Ming Fu | Advisor: Jeff Schneider (CMU) | Sponsor: Argo AI","publisher":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#organization","name":"Reinforcement Learning for Self Driving Cars","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#\/schema\/logo\/image\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/04\/cropped-car-4343634-scaled-1.jpg","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-content\/uploads\/sites\/46\/2022\/04\/cropped-car-4343634-scaled-1.jpg","width":288,"height":250,"caption":"Reinforcement Learning for Self Driving Cars"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/pages\/22","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/users\/98"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/comments?post=22"}],"version-history":[{"count":11,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/pages\/22\/revisions"}],"predecessor-version":[{"id":97,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/pages\/22\/revisions\/97"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teama\/wp-json\/wp\/v2\/media?parent=22"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}