Leveraging uncertainty information to refine semantic segmentation for port and marine structures in the CNN context

Previously, several segmentation networks have been trained to detect damaged areas on port infrastructure. These models are optimized variants of the Deeplab V3+ architecture (fourth version of Google's deep learning model for image segmentation) and consists of different feature extraction networks. Their individual predictions can be combined in an ensemble approach to achieve a more robust segmentation result. This provides for example the possibility to compute the standard deviation as an uncertainty measure. Of course, in the literature are several other procedures available as Monte Carlo Dropout or Variational Inference for Bayesian Networks. Since we do not expect significantly performance benefits due to further architectural improvements itself, we want to provide additional information to the network. In this thesis, you shall investigate the benefit when uncertainty information is provided to a new instantiated segmentation model. Because the main purpose of the current trained network is to assist infrastructure inspectors, the model shall also be able to predict it's own uncertainty at the end. This gives an operator an indication whether a certain image section or segment has to be corrected manually. Currently, the considered dataset for training and testing shows the quay wall of a German port partially in 70 labeled images. The main present damage type is corrosion, which is the reason that the models can only distinguish this class from the background so far. The image pairs are split into (overlapping) patches of 512x512 pixels to increase the number of samples to feed through the segmentation network.