{"id":13,"date":"2021-12-10T02:40:52","date_gmt":"2021-12-10T02:40:52","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/?page_id=13"},"modified":"2022-05-01T21:47:48","modified_gmt":"2022-05-01T21:47:48","slug":"method","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/","title":{"rendered":"Method"},"content":{"rendered":"\n<p class=\"has-large-font-size\">Framework<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Basline<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"387\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1024x387.png\" alt=\"\" class=\"wp-image-188\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1024x387.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-300x113.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-768x290.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1536x580.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-2048x773.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><figcaption>Baseline Model.<\/figcaption><\/figure>\n\n\n\n<p>The framework we use is based on the Transfuser [1] model, which contains two parts (1) <strong>a Multi-Modal Fusion Transformer <\/strong>for integrating information from multiple modalities (single-view image from the frontal camera and LiDAR in this case), (2) <strong>an auto-regressive waypoint prediction network<\/strong> for predicting the discretized future trajectory of the self-driving vehicle.  <\/p>\n\n\n\n<p>The supervision of the baseline model is designed as the L1 loss between the predicted waypoints and the ground truth waypoints:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-3.png\" alt=\"\" class=\"wp-image-183\" width=\"138\" height=\"46\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-3.png 698w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-3-300x101.png 300w\" sizes=\"auto, (max-width: 138px) 100vw, 138px\" \/><\/figure>\n\n\n\n<p class=\"has-white-color has-text-color\">1<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Auxiliary Freespace Forecasting Task<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"451\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-1024x451.png\" alt=\"\" class=\"wp-image-159\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-1024x451.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-300x132.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-768x338.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-1536x676.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/framework-2048x901.png 2048w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><figcaption>Proposed Model: Introducing auxiliary freespace forecasting task.<\/figcaption><\/figure>\n\n\n\n<p>Inspired by [2], we introduced <strong>freesapce forecasting <\/strong>as an auxiliary task and designed an additional <span class=\"has-inline-color has-vivid-red-color\">freespace forecasting decoder<\/span> (shown in the red block above), using the fused feature from LiDAR branch before the final average polling layer, because the features in this layer are fully extracted and at proper size. The freespace decoder consists of 4 layers of Upsampling-Conv-ReLU combination and another Upsampling-Conv layer (shown in the bottom figure), following simplicity principle in order not to dominate the main waypoints prediction task.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-5-1024x690.png\" alt=\"\" class=\"wp-image-193\" width=\"312\" height=\"211\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-5-1024x690.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-5-300x202.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-5-768x518.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-5.png 1166w\" sizes=\"auto, (max-width: 312px) 100vw, 312px\" \/><figcaption>Freespace Decoder.<\/figcaption><\/figure><\/div>\n\n\n\n<p>Our intuition behind adding the assistant freespace forecasting task is that, the original loss function, which is a simple waypoints deviation punishment restricting the L1 distance between the ground waypoints and predicted waypoints is too weak. Consider the <strong>spareness<\/strong> of the waypoints loss (only several points) compared with the dense RGB and LiDAR input (2D matrix or even 3D tensor if taking history as input). It is highly likely that even if the model learns from ground truth after thousands of epoches, it just learns similar behaviors instead of the reasons behind.<\/p>\n\n\n\n<p>Another reason why we introduced freespace forecasting is becasue it is <strong>self-supervised<\/strong>. Given a Bird Eye View LiDAR frame, the freespace can be directly computed by raycasting. Consequently, we <strong>do not need expensive human annotations<\/strong> to get ground truth freespace, which makes it convenient to add the task. The following is an example of freespace forecasting centering at the self-driving vehicle, where <span class=\"has-inline-color has-vivid-red-color\">Red<\/span> denotes vehicles and <span class=\"has-inline-color has-vivid-green-cyan-color\">Green<\/span> denotes freespace. The corresponding timestamps from left to right are t, t+1 and t+2.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"202\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6-1024x202.png\" alt=\"\" class=\"wp-image-198\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6-1024x202.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6-300x59.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6-768x151.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6-1536x303.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-6.png 2000w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><figcaption>Example of freespace forecasting. <span class=\"has-inline-color has-vivid-red-color\">Red<\/span>: Vehicles, <span class=\"has-inline-color has-vivid-green-cyan-color\">Green<\/span>: Freespace. From left to right timestamps: t, t+1, t+2.<\/figcaption><\/figure>\n\n\n\n<p>For freespace forecasting, we use Binary Cross Entropy as supervision, and the total loss is the weighted combination of wapoints prediction loss and the freespace forecasting loss. In our experiments, we set alpha equals to 1.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-7-1024x777.png\" alt=\"\" class=\"wp-image-202\" width=\"171\" height=\"130\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-7-1024x777.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-7-300x228.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-7-768x583.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-7.png 1210w\" sizes=\"auto, (max-width: 171px) 100vw, 171px\" \/><\/figure>\n\n\n\n<p class=\"has-white-color has-text-color\">1<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Learning to Predict Future Cost Map<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"394\" src=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image-1024x394.png\" alt=\"\" class=\"wp-image-229\" srcset=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image-1024x394.png 1024w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image-300x115.png 300w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image-768x296.png 768w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image-1536x591.png 1536w, https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2022\/05\/image.png 1720w\" sizes=\"auto, (max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><figcaption>Proposed Model: Learning to predict future cost map.<\/figcaption><\/figure>\n\n\n\n<p>In order to predict cost maps, we introduced <mark class=\"has-inline-color has-vivid-red-color\">costmap decoder<\/mark> (shown in the blue block above), using the fused feature from LiDAR branch before the final average polling layer. The predicted cost maps will be used to optimize the initialized trajectory to obtain the final trajectory.<\/p>\n\n\n\n<p class=\"has-medium-font-size\">Cost Map Settings<\/p>\n\n\n\n<p>Waypoint cost maps. Waypoints cost maps cover the area of the correct lane and direction, which aim to guide the vehicle to follow the lane exactly. <\/p>\n\n\n\n<p><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh4.googleusercontent.com\/1vMfy_RNQPXTmceSnBXSj1smmgL_6X6fa6mYn_13w0AuLisLVUUVJIohyiV-WSE9KF6oPqsWJNNJeNSmPhQNGihUXO9HVyPWwW20wdTvF-MugxmNuA7eJeD6B6OeKD99OHodv7g1HXGzn0c2VzZncA\"><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh5.googleusercontent.com\/BAEG7UnHCE4z0ZI_HkynCXBAqB1JXEUBKtvErjSOz0_p7tm3FD3M3F6pXefoYynTHIaIY3o3JiJ0tSzoenLcSRJSpjegNnVGiVVOU4TVgL2ldhwYXAKyqO-Lv40TI-T76CsJ8IwyZ2Yg7MDhAwbNIA\"><\/p>\n\n\n\n<p><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh5.googleusercontent.com\/taR2Li-sUyDFrPvvu5t8vouc3k-Ai1mRxnhM5MMnMDOKv3akuj4cJpsQc4YTEchlubuQ-nqsOQyFBQ7B1WUO-LtFg4b9KLHNtgB2wHRlP1FGtCc6Axnnydtp4y0Chetcih5FvI-j94431zrw9wVE-Q\"><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh4.googleusercontent.com\/q3lC92tnYq-QLE0SIebiKGvVhFRMi68IG5mXgiHVONWE0z0_S54CDk5AS-gczT55cFQjXNOisMx2W9raZD9Nn2y1EdBMMXM-DV21zYDku_xhyYGZp6G8mXk423NYAfz21A_MJflb84S-Xy5vu3t5Hg\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>Collision cost maps. Collision cost maps refer to all other vehicles and pedestrians, playing the role of collision avoidance. <\/p>\n\n\n\n<p><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh4.googleusercontent.com\/fkjh2JuNf1uOT4VtOEO2uiR69_TNuYFFlMgsw5bXmO5src_HVmB4IT1ik4SyQkhM60q5QTAxmA0wpNREuYTW05CTT3sNWUAftnEJy0Hk-fvyE1JieK_MOLa1wuOHBy1WLQj6LY7v3fvSVIBu7cmDZw\"><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh6.googleusercontent.com\/5SEmETGhV5_nHDXasdpBzG4dqmb8UuJ9K4ovIxZRXPxFhjiGc0wSBnt1S-T55mYui0Pl-4WmKZtvh7f4VQ16Mooyj1A0vz1JeKTvu0ttUo-keGvblAnSPheGGM118NF3a5K7Rt1eloiPiJkz6lI2xA\"><\/p>\n\n\n\n<p><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh5.googleusercontent.com\/3-tBwY_wiii1JXWOBh0xqIxtvAjoY2OIspZ0pm71umilTG0qUbrWlU0MGHLeS_hn1I2ZlKQ_WOc3nMBUCZelAFvJ1yYxX_q6K66A1SIxEkG1CvdMkVP8tD-UzO3cgRqo-ojU6UmtH1R0GAuAPnHB6Q\"><img decoding=\"async\" width=\"175px;\" height=\"175px;\" src=\"https:\/\/lh6.googleusercontent.com\/gWJFMYbWGwTE1xBb7s4vj3-wvOAxLbc07k-syrhlao3lZc0ZIsYvIZhJe1uHk-dxLKtl810dEsEQz9Nxt_jiccAOXLMp60pyvJU-vnn34i-RC8dPaxZeKtlYvsQXzkyXFW03TmsSOjqUHNOmK-xXjg\"><\/p>\n\n\n\n<p>Traffic sign cost maps. Traffic sign cost maps are defined as an instant rectangle area in front of the self vehicle, indicating the status of the traffic light that is currently affecting the self vehicle. <\/p>\n\n\n\n<p><img decoding=\"async\" width=\"209px;\" height=\"209px;\" src=\"https:\/\/lh5.googleusercontent.com\/vAV9DIeF1RzLohS8-xQadQVTm7GENrkXKoV66qdYCUkc4u8rzK4yFdK8_0e7LCSu40KIP-k0ZlwMoV_gUmW4_kRDwsknHyzTKC4vKMACOpHWy35mG2rVPaLHpHly4bheFmSi1-3N-1H2K1pGOrdEgA\"><img decoding=\"async\" width=\"209px;\" height=\"209px;\" src=\"https:\/\/lh6.googleusercontent.com\/9GGRk9Th62BUDgYqebU4nniBaJYEcyQwuyRSxyyHmYAeA6M6xzUbpLcq-0ycmJ7fCfW-s5mUuFS-Og9P0dJldq5QDbSQEoh-_gtv4YHSNJxtrC2JZx7qo-dDCuoVcmNIGcUdvOwGeRUc4gu2eQunnA\"><img decoding=\"async\" width=\"209px;\" height=\"209px;\" src=\"https:\/\/lh3.googleusercontent.com\/zznryghM5qGgDvJXRrigj_miXCzCrCpyli2qd7r92BJuuLRf6zdawQMev_gQGstim-XGBFO7Uae_glabrTNXM8BrQ0YzHUfUcrIX0MKqlj_qc23HPxXN5vklEB2nQNdL6PxleV9RnW_ySB9pIgZf9Q\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p class=\"has-medium-font-size\">Reference<\/p>\n\n\n\n<p class=\"has-small-font-size\">[1] Prakash Aditya. Multi-Modal Fusion Transformer for End-to-End Autonomous Driving. CVPR 2021.  Link: <a href=\"https:\/\/arxiv.org\/pdf\/2104.09224.pdf\">https:\/\/arxiv.org\/pdf\/2104.09224.pdf<\/a><\/p>\n\n\n\n<p class=\"has-small-font-size\">[2]. Hu, Peiyun, et al. &#8220;Safe Local Motion Planning with Self-Supervised Freespace Forecasting.&#8221; <em>CVPR, <\/em>2021. Link: <a href=\"https:\/\/openaccess.thecvf.com\/content\/CVPR2021\/papers\/Hu_Safe_Local_Motion_Planning_With_Self-Supervised_Freespace_Forecasting_CVPR_2021_paper.pdf\">https:\/\/openaccess.thecvf.com\/content\/CVPR2021\/papers\/Hu_Safe_Local_Motion_Planning_With_Self-Supervised_Freespace_Forecasting_CVPR_2021_paper.pdf<\/a><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Framework Basline The framework we use is based on the Transfuser [1] model, which contains two parts (1) a Multi-Modal Fusion Transformer for integrating information from multiple modalities (single-view image from the frontal camera and LiDAR in this case), (2) an auto-regressive waypoint prediction network for predicting the discretized future trajectory of the self-driving vehicle. &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Method&#8221;<\/span><\/a><\/p>\n","protected":false},"author":105,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-13","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Method - Forecasting for Autonomous Driving<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Method - Forecasting for Autonomous Driving\" \/>\n<meta property=\"og:description\" content=\"Framework Basline The framework we use is based on the Transfuser [1] model, which contains two parts (1) a Multi-Modal Fusion Transformer for integrating information from multiple modalities (single-view image from the frontal camera and LiDAR in this case), (2) an auto-regressive waypoint prediction network for predicting the discretized future trajectory of the self-driving vehicle. &hellip; Continue reading &quot;Method&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/\" \/>\n<meta property=\"og:site_name\" content=\"Forecasting for Autonomous Driving\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-01T21:47:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1024x387.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/\",\"name\":\"Method - Forecasting for Autonomous Driving\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image-4-1024x387.png\",\"datePublished\":\"2021-12-10T02:40:52+00:00\",\"dateModified\":\"2022-05-01T21:47:48+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image-4.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/wp-content\\\/uploads\\\/sites\\\/49\\\/2021\\\/12\\\/image-4.png\",\"width\":2294,\"height\":866},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/method\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Method\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/\",\"name\":\"Forecasting for Autonomous Driving\",\"description\":\"Xiaochen Han &amp; Tanay Sharma | Prof. David Held &amp; Prof. Deva Ramanan (Argo)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamd\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Method - Forecasting for Autonomous Driving","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/","og_locale":"en_US","og_type":"article","og_title":"Method - Forecasting for Autonomous Driving","og_description":"Framework Basline The framework we use is based on the Transfuser [1] model, which contains two parts (1) a Multi-Modal Fusion Transformer for integrating information from multiple modalities (single-view image from the frontal camera and LiDAR in this case), (2) an auto-regressive waypoint prediction network for predicting the discretized future trajectory of the self-driving vehicle. &hellip; Continue reading \"Method\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/","og_site_name":"Forecasting for Autonomous Driving","article_modified_time":"2022-05-01T21:47:48+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1024x387.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/","name":"Method - Forecasting for Autonomous Driving","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4-1024x387.png","datePublished":"2021-12-10T02:40:52+00:00","dateModified":"2022-05-01T21:47:48+00:00","breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/49\/2021\/12\/image-4.png","width":2294,"height":866},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/method\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/"},{"@type":"ListItem","position":2,"name":"Method"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/","name":"Forecasting for Autonomous Driving","description":"Xiaochen Han &amp; Tanay Sharma | Prof. David Held &amp; Prof. Deva Ramanan (Argo)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/13","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/comments?post=13"}],"version-history":[{"count":15,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/13\/revisions"}],"predecessor-version":[{"id":233,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/13\/revisions\/233"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/media?parent=13"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}