Performance Comparison of Backbone Networks for Multi-Tasking in Self-Driving Operations

In the era of big data, increased focus has been on improving neural network based Deep Learning models. This led to various classification networks which can be used as a backbone in multi-task learning. However, depending on the selected backbone, multi-tasking performance differs. While given backbone network shows better performance on a detection task, does not mean such performance generalizes in segmentation task as well. Detailed investigations should be conducted to achieve best inference speed-accuracy trade-off prior to implementing a single neural network, which handles multiple tasks. In this research, the performance comparison among EfficientNet, ResNet101, VGG16, ResNet50 and MobilenetV2 on the Berkeley Driving Dataset (BDD100K) for autonomous driving using multi-tasking architecture are provided. Backbones that offer best time-accuracy trade-off for multi-task learning are evaluated. Implemented architecture contains three most crucial tasks in self-driving operations, object detection, drivable area segmentation and lane detection. EfficientNet based model showed the best mAP on the object detection task, as well as on the segmentation tasks, extracting both the long and wide roads with accurate lane lines. The model with MobilenetV2 backbone however, demonstrates the fastest inference speed with relatively lower performance in all tasks.