{"id":63,"date":"2021-12-10T02:42:18","date_gmt":"2021-12-10T02:42:18","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/?page_id=63"},"modified":"2022-05-01T20:43:42","modified_gmt":"2022-05-01T20:43:42","slug":"results","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/","title":{"rendered":"Results"},"content":{"rendered":"\n

Simulation on CARLA simulator<\/p>\n\n\n\n

Dataset<\/p>\n\n\n\n

We use the famous CARLA[1] dataset for both training and evalution. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites, environmental conditions, full control of all static and dynamic actors, maps generation and much more.<\/p>\n\n\n\n

Experimental Setup<\/p>\n\n\n\n

We follow the experimental setup of Transfuser, trained the Transfuser and our proposed Transfuser w\/ ff on the same training dataset, which is generated using CARLA simulator, provided by the authors of Transfuser. We evaluate both methods using the same configuration on the CARLA simulator.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Methods<\/p>\n\n\n\n

Transfuser<\/strong>: The original Transfuser baseline.<\/p>\n\n\n\n

Transfuser w\/ ff<\/strong> : Our proposed Transfuser with auxiliary freespace forecasting task.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Visualization results<\/p>\n\n\n\n

(1): Turning left when there is a motor rider. Transfuser got blocked; Transfuser w\/ ff avoided being blocked<\/strong>.<\/p>\n\n\n\n

Transfuser link: https:\/\/youtu.be\/URAvD2X9FHI<\/a><\/p>\n\n\n\n

Transfuser w\/ ff link: https:\/\/youtu.be\/0_mTwB4eKH8<\/a><\/p>\n\n\n\n

(2): Normally going in front. Transfuser just stopped without reasons; Transfuser w\/ ff went through as common<\/strong>.<\/p>\n\n\n\n

Transfuser link: https:\/\/youtu.be\/L9QWdc9jG-U<\/a><\/p>\n\n\n\n

Transfuser w\/ ff link: https:\/\/youtu.be\/Z5W5i3PIxYI<\/a><\/p>\n\n\n\n

(3): Facing dangerous behaviors of others. Transfuser crashed with another car and ran over the person; Transfuser w\/ ff successfully detected the behaviors and stopped before crashing<\/strong>.<\/p>\n\n\n\n

Transfuser link: https:\/\/youtu.be\/rCyCmf-3Mpg<\/a><\/p>\n\n\n\n

Transfuser w\/ ff link: https:\/\/youtu.be\/N5icm2Vdo5c<\/a><\/p>\n\n\n\n

<\/p>\n\n\n\n

Experimental results<\/p>\n\n\n\n

<\/td>Driving Score<\/td>Route Completion<\/td>Infraction Penalty<\/td><\/tr>
Transfuser<\/td>42.563<\/td>90.688<\/td>0.512<\/td><\/tr>
Transfuser w\/ ff<\/td>45.781<\/strong><\/td>87.773<\/td>0.554<\/strong><\/td><\/tr><\/tbody><\/table>
Main Metrics. Higher is better.<\/figcaption><\/figure>\n\n\n\n

Driving Score<\/strong>: The main metric, serving as an aggregate of the average route completion and the number of traffic infractions.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n\n\n\n

1<\/p>\n\n\n\n

1<\/p>\n\n\n\n

Route Completion<\/strong>: Percentage of route distance completed by an agent.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n\n\n\n

1<\/p>\n\n\n\n

1<\/p>\n\n\n\n

Infraction Penalty<\/strong>: Aggregates the number of infractions triggered by an agent as a geometric series.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Average num of infractions<\/td>Pedestrians<\/td>Vehicles<\/td>Layouts<\/td>Red Lights<\/td><\/tr>
Transfuser<\/td>0.125<\/td>0.438<\/td>0.063<\/td>1.594<\/td><\/tr>
Transfuser w\/ ff<\/td>0<\/strong><\/td>0.406<\/strong><\/td>0<\/strong><\/td>1.469<\/strong><\/td><\/tr><\/tbody><\/table>
Average Number of different kinds of Infractions. Lower is better.<\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

Conclusion<\/p>\n\n\n\n

With additional freespace forecasting, our model improves the general driving score and provides safer local motion planning compared with the baseline model. In our further analysis, it as well decreases the collion rate with pedestrians, layouts and vehicles. But freespace forecasting does little assistance with red lights detections, as the freespace takes more from LiDAR data rather than RGB images, from which the traffic lights can be detected. We would need to explore more efficient way to use RGB information.<\/p>\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n\n\n\n

Cost Map Prediction<\/p>\n\n\n\n

Visualization results<\/p>\n\n\n\n

General results<\/p>\n\n\n\n

<\/p>\n\n\n\n

Waypoints cost map results<\/p>\n\n\n\n

<\/p>\n\n\n\n

Collision cost map results<\/p>\n\n\n\n

<\/p>\n\n\n\n

Conclusion<\/p>\n\n\n\n

For waypoint cost map prediction, the short distance accuracy is acceptable, but it is not trivial for the model to figure out the correct direction if there is a turning point in the future. For collision cost map prediction, the model can locate the main objects nearby such as vehicles correctly, but sometimes it ignores small targets like pedestrians or people who are blocked by other vehicles.<\/p>\n\n\n\n

Still need to improve the prediction quality.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Reference<\/p>\n\n\n\n

[1]. Dosovitskiy, Alexey, et al. “CARLA: An open urban driving simulator.” PMLR, 2017.<\/p>\n","protected":false},"excerpt":{"rendered":"

Simulation on CARLA simulator Dataset We use the famous CARLA[1] dataset for both training and evalution. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this … <\/p>\n

Continue reading “Results”<\/span><\/a><\/p>\n","protected":false},"author":105,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-63","page","type-page","status-publish","hentry"],"yoast_head":"\nResults - Forecasting for Autonomous Driving<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Results - Forecasting for Autonomous Driving\" \/>\n<meta property=\"og:description\" content=\"Simulation on CARLA simulator Dataset We use the famous CARLA[1] dataset for both training and evalution. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this … Continue reading "Results"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/\" \/>\n<meta property=\"og:site_name\" content=\"Forecasting for Autonomous Driving\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-01T20:43:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/\",\"name\":\"Results - Forecasting for Autonomous Driving\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image.png\",\"datePublished\":\"2021-12-10T02:42:18+00:00\",\"dateModified\":\"2022-05-01T20:43:42+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image.png\",\"width\":422,\"height\":190},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/results\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Results\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/\",\"name\":\"Forecasting for Autonomous Driving\",\"description\":\"Xiaochen Han & Tanay Sharma | Prof. David Held & Prof. Deva Ramanan (Argo)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Results - Forecasting for Autonomous Driving","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/","og_locale":"en_US","og_type":"article","og_title":"Results - Forecasting for Autonomous Driving","og_description":"Simulation on CARLA simulator Dataset We use the famous CARLA[1] dataset for both training and evalution. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this … Continue reading \"Results\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/","og_site_name":"Forecasting for Autonomous Driving","article_modified_time":"2022-05-01T20:43:42+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/","name":"Results - Forecasting for Autonomous Driving","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image.png","datePublished":"2021-12-10T02:42:18+00:00","dateModified":"2022-05-01T20:43:42+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image.png","width":422,"height":190},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/results\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/"},{"@type":"ListItem","position":2,"name":"Results"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/","name":"Forecasting for Autonomous Driving","description":"Xiaochen Han & Tanay Sharma | Prof. David Held & Prof. Deva Ramanan (Argo)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/63","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/comments?post=63"}],"version-history":[{"count":19,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/63\/revisions"}],"predecessor-version":[{"id":226,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/63\/revisions\/226"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/media?parent=63"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}