Results

Simulation on CARLA simulator

Dataset

We use the famous CARLA[1] dataset for both training and evalution. CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites, environmental conditions, full control of all static and dynamic actors, maps generation and much more.

Experimental Setup

We follow the experimental setup of Transfuser, trained the Transfuser and our proposed Transfuser w/ ff on the same training dataset, which is generated using CARLA simulator, provided by the authors of Transfuser. We evaluate both methods using the same configuration on the CARLA simulator.

Methods

Transfuser: The original Transfuser baseline.

Transfuser w/ ff : Our proposed Transfuser with auxiliary freespace forecasting task.

Visualization results

(1): Turning left when there is a motor rider. Transfuser got blocked; Transfuser w/ ff avoided being blocked.

Transfuser link: https://youtu.be/URAvD2X9FHI

Transfuser w/ ff link: https://youtu.be/0_mTwB4eKH8

(2): Normally going in front. Transfuser just stopped without reasons; Transfuser w/ ff went through as common.

Transfuser link: https://youtu.be/L9QWdc9jG-U

Transfuser w/ ff link: https://youtu.be/Z5W5i3PIxYI

(3): Facing dangerous behaviors of others. Transfuser crashed with another car and ran over the person; Transfuser w/ ff successfully detected the behaviors and stopped before crashing.

Transfuser link: https://youtu.be/rCyCmf-3Mpg

Transfuser w/ ff link: https://youtu.be/N5icm2Vdo5c

Experimental results

Driving ScoreRoute CompletionInfraction Penalty
Transfuser42.56390.6880.512
Transfuser w/ ff45.78187.7730.554
Main Metrics. Higher is better.

Driving Score: The main metric, serving as an aggregate of the average route completion and the number of traffic infractions.

1

1

Route Completion: Percentage of route distance completed by an agent.

1

1

Infraction Penalty: Aggregates the number of infractions triggered by an agent as a geometric series.

Average num of infractionsPedestriansVehiclesLayoutsRed Lights
Transfuser0.1250.4380.0631.594
Transfuser w/ ff00.40601.469
Average Number of different kinds of Infractions. Lower is better.

Conclusion

With additional freespace forecasting, our model improves the general driving score and provides safer local motion planning compared with the baseline model. In our further analysis, it as well decreases the collion rate with pedestrians, layouts and vehicles. But freespace forecasting does little assistance with red lights detections, as the freespace takes more from LiDAR data rather than RGB images, from which the traffic lights can be detected. We would need to explore more efficient way to use RGB information.

Cost Map Prediction

Visualization results

General results

Waypoints cost map results

Collision cost map results

Conclusion

For waypoint cost map prediction, the short distance accuracy is acceptable, but it is not trivial for the model to figure out the correct direction if there is a turning point in the future. For collision cost map prediction, the model can locate the main objects nearby such as vehicles correctly, but sometimes it ignores small targets like pedestrians or people who are blocked by other vehicles.

Still need to improve the prediction quality.

Reference

[1]. Dosovitskiy, Alexey, et al. “CARLA: An open urban driving simulator.” PMLR, 2017.