CNN-based augmentation strategy for spectral unmixing datasets considering spectral variability

Spectral unmixing is often relying on a mixing model that is only an approximation. Artificial neural networks have the advantage of not requiring model knowledge. Additional advantages in the domain of spectral unmixing are the easy handling of spectral variability and the possibility to force the sum-to-one and the non-negativity constraints. However, they need a lot of significant training data to achieve good results. To overcome this problem, mainly for classification problems, augmentation strategies are widely used to increase the size of training datasets synthetically. Spectral unmixing can be considered as a regression problem, where data augmentation is also feasible. One intuitive strategy is to generate spectra based on abundances that do not occur in the training dataset, while taking spectral variability into account. For the implementation of this approach, we use a convolutional neural network (CNN), where the input variables are extended by random values. This allows spectral variability to be taken into account. The random inputs are re-sampled for each data point in every epoch. During training the CNN learns the mixing model and the characteristic spectral variability of the training dataset. Additional spectra can be generated afterwards for any given abundances to extend the original training dataset. Because the generative CNN minimizes the error between generated spectra and the corresponding ground truth for the whole dataset during training, the variance of the spectra based on the same abundances is lower than in the training data. We have investigated two approaches for improvement. One is to increase the variance of the random input variables when generating new spectra. For the second, the estimated covariance matrices are considered by the objective function. The presented method is evaluated with real data, which were captured in our image processing laboratory. We found that the augmentation of the training dataset with the presented strategy leads to an improvement for spectral unmixing of the test dataset.