{"id":9,"date":"2019-05-05T19:37:38","date_gmt":"2019-05-05T19:37:38","guid":{"rendered":"http:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/?page_id=9"},"modified":"2019-12-06T16:16:18","modified_gmt":"2019-12-06T21:16:18","slug":"project-summary","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/","title":{"rendered":"Project Summary"},"content":{"rendered":"

Abstract<\/h1>\n

Autonomous driving presents one of the largest
\nproblems that the robotics and artificial intelligence communities
\nare facing at the moment, both in terms of difficulty
\nand potential societal impact. Self-driving vehicles (SDVs) are
\nexpected to prevent road accidents and save millions of lives
\nwhile improving the livelihood and life quality of many more.
\nHowever, despite large interest and several companies working in the autonomous domain, there is still more
\nto be done to develop a system capable of operating
\nat a level comparable to the best human drivers. One reason for
\nthis is high uncertainty of traffic behavior and a large number of
\nsituations that an SDV may encounter on the roads, making it
\nvery difficult to create a fully generalizable system. To ensure
\nsafe and efficient operations, an autonomous vehicle is required
\nto account for this uncertainty and to anticipate a multitude of
\npossible behaviors of traffic actors in its surrounding.<\/p>\n

\"\"<\/p>\n

In this project, we discuss an important topic and address one of the crucial aspects, the problem of predicting the future state of an SDV’s surroundings for safe and efficient operations. We are working with Prof. Jeff Schneider to develop deep-learning-based approaches that take into account the current world state to infer future movement of actors (vehicles, pedestrians, bicyclists).<\/p>\n

\"\"<\/p>\n

Detailed Information<\/h1>\n

Part (a): Vulnerable Road Users<\/h2>\n

Vulnerable Road Users (VRUs) are traffic actors with an increased risk of injury, generally unprotected by an outside shield. The two main actor classes under VRUs are pedestrians and bicyclists. We focus our ideas on pedestrians, as they account for a much larger proportion of the VRUs. We set up baselines on the BIWI and UCY datasets using several approaches. In particular, we experiment with Social-LSTM, Social-GAN, and LVA-LSTM.<\/p>\n

Social-LSTM takes into account neighborhood interactions through a social pooling layer. Social-GAN extends the idea of Social-LSTM by adding scene level interactions, using a GAN to emulate more natural trajectories and using a Variety Loss to help predict the best of multiple sampled trajectories. LVA-LSTM encodes velocity information along with locations and uses attention to model the future velocities and locations of the pedestrians.<\/p>\n

More information on the models can be seen in the final presentation slides.<\/p>\n

\"\"<\/h2>\n

Part (b): Vehicles<\/h2>\n

Vehicles make up for the largest proportion of all actors for an SDV. Towards this, we set up a baseline on the KITTI autonomous dataset using several approaches. In particular, we experiment with two state-of-art models, namely, DESIRE-SI and INFER-Skip which are stochastic and deterministic respectively.<\/p>\n

DESIRE-SI combines the information of agent interactions, scene context, and velocity to provide a very detailed context for predictions. To predict multi-modal paths, it uses a Conditional Variational Auto-Encoder and uses Inverse Optimal Control to course-correct its predictions. INFER, on the other hand, builds intermediate representations and uses ConvLSTMs to predict the future state of actors as heatmaps.<\/p>\n

More information on the models can be seen in the final presentation slides.<\/p>\n

\"\"<\/p>\n

Argoverse<\/h1>\n

Argoverse<\/a> is a recently released dataset by Argo AI<\/a>. We focus on modeling the trajectories in both regular (xy) coordinates and also in the centerline (nt) coordinates, as specified for the AGENT class. We work in two directions – recurrent and feedforward.<\/p>\n

Part (a): Recurrent Modeling<\/h2>\n

Here we focus on two models: Vanilla LSTM encoder-decoders and Social-LSTM — which does social pooling similar to Social-GAN mentioned above.<\/p>\n

\"\"<\/p>\n

Part (a): Feedforward Modeling<\/h2>\n

Inspired by recent work on feedforward sequential modeling in the Language domain, we work on recent architectures such as Temporal Convolutional Networks (TCN), Trellis Networks (weight-tied, input-injected TCNs) and Deep Equilibrium Models (root-finding for optimizing weight-tied networks towards an equilibrium point using implicit differentiation of Quasi-Newton methods).<\/p>\n

\"\"<\/p>\n","protected":false},"excerpt":{"rendered":"

Abstract Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, … <\/p>\n

Continue reading “Project Summary”<\/span><\/a><\/p>\n","protected":false},"author":46,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-9","page","type-page","status-publish","hentry"],"yoast_head":"\nProject Summary - Deep Prediction for Self-Driving Cars<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Project Summary - Deep Prediction for Self-Driving Cars\" \/>\n<meta property=\"og:description\" content=\"Abstract Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, … Continue reading "Project Summary"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/\" \/>\n<meta property=\"og:site_name\" content=\"Deep Prediction for Self-Driving Cars\" \/>\n<meta property=\"article:modified_time\" content=\"2019-12-06T21:16:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-content\/uploads\/sites\/22\/2019\/05\/Picture1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"356\" \/>\n\t<meta property=\"og:image:height\" content=\"216\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/\",\"name\":\"Project Summary - Deep Prediction for Self-Driving Cars\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/wp-content\\\/uploads\\\/sites\\\/22\\\/2019\\\/05\\\/Picture1-300x182.png\",\"datePublished\":\"2019-05-05T19:37:38+00:00\",\"dateModified\":\"2019-12-06T21:16:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/wp-content\\\/uploads\\\/sites\\\/22\\\/2019\\\/05\\\/Picture1.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/wp-content\\\/uploads\\\/sites\\\/22\\\/2019\\\/05\\\/Picture1.png\",\"width\":356,\"height\":216},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/project-summary\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Project Summary\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/\",\"name\":\"Deep Prediction for Self-Driving Cars\",\"description\":\"Carnegie Mellon University\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2019teamg\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Project Summary - Deep Prediction for Self-Driving Cars","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/","og_locale":"en_US","og_type":"article","og_title":"Project Summary - Deep Prediction for Self-Driving Cars","og_description":"Abstract Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, … Continue reading \"Project Summary\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/","og_site_name":"Deep Prediction for Self-Driving Cars","article_modified_time":"2019-12-06T21:16:18+00:00","og_image":[{"width":356,"height":216,"url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-content\/uploads\/sites\/22\/2019\/05\/Picture1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/","name":"Project Summary - Deep Prediction for Self-Driving Cars","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/#primaryimage"},"thumbnailUrl":"http:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-content\/uploads\/sites\/22\/2019\/05\/Picture1-300x182.png","datePublished":"2019-05-05T19:37:38+00:00","dateModified":"2019-12-06T21:16:18+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-content\/uploads\/sites\/22\/2019\/05\/Picture1.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-content\/uploads\/sites\/22\/2019\/05\/Picture1.png","width":356,"height":216},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/project-summary\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/"},{"@type":"ListItem","position":2,"name":"Project Summary"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/","name":"Deep Prediction for Self-Driving Cars","description":"Carnegie Mellon University","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/pages\/9","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/users\/46"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/comments?post=9"}],"version-history":[{"count":8,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/pages\/9\/revisions"}],"predecessor-version":[{"id":102,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/pages\/9\/revisions\/102"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2019teamg\/wp-json\/wp\/v2\/media?parent=9"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}