This post describes a student group project developed within the Data Science Lab undergraduate course of the Vienna University of Economics and Business, co-supervised by TIMETOACT GROUP Austria.
Student project team: Michael Fixl, Josef Hinterleitner, Felix Krause and Adrian Seiß
Supervisors: Prof. Dr. Axel Polleres (WU Vienna), Dr. Vadim Savenkov (Trustbit)
Introduction
In the previous blog post (which you can find here), we created an already pretty powerful image segmentation model in order to detect the shape of truck parking lots on satellite images. However, we will now try to run the code on new hardware and get even better as well as more robust results.
New hardware and out-of-sample data
Up to now we trained all semantic segmentation models on Google Colab, which is advantageous as this assures everyone can reproduce the project easily but unfortunately the platform partly uses old package versions. In order to check out if the model training can simply be repeated on different hardware, we train the promising semantic segmentation models once again but with the newest package versions and on a high-end computer with an RTX 3090 graphics card. Hereby we can see that there can in fact be some randomness in the results depending on the hardware and packages used.
Let’s also check how robust our model is against more varied data, e.g. coming in different scales or from sources other than OpenStreetMap. Therefore, we made some experiments with unseen truck parking lots from Google Earth with various parking lot positions and sizes. Checking those images with the optimized PSPNet model from the last blog post shows that generalizability is not yet convincing, as you can see below.
Further testing and optimization
To counter the above mentioned issues, we should implement some fixes to reduce the risk of overfitting and check generalizability of the other models. Interestingly, further testing and optimization now shows that the LinkNet architecture turns out to be very precise and much more generalizable than PSPNet. It reaches a mIoU of up to 75% in the test set while also achieving high precision on completely new data with different structure than the train data.
You can find some predictions of this model on unseen parking lots from Google Earth below. Also this model of course fails under certain conditions and does not predict anything or predicts falsely as you can see in the right sample image. If you are interested in more detailed results, you might check out the comparison of the model performance before and after the previously mentioned changes in the table at the end of the blog post. There you can also find the training process of the final LinkNet model.
Conclusion
Overall, in this narrowly defined task the optimized LinkNet model shows decent accuracy and generalizability. Instead of sending expensive workers on site to measure coordinates of unrecorded truck parking lots, this model could be used to get a sometimes rough but at the same time very cheap estimation of the shapes of parking areas. Knowing these, the current activity of a truck can be derived based on GPS signals. This information could then be used to optimize transport operations.
Finally, a small app concludes the project, which is visible in the screenshot above. It simply takes coordinates as input and outputs a prediction mask using the optimized LinkNet model from before.
Additional material
Testing results
*new hardware, code optimization and changes (overfitting fix, newest package versions, etc.)
Training process of final LinkNet model
