DGSSC A Deep Generative Spectral Spatial Classifier for Imbalanced Hyperspectral Imagery

In recent years, hyperspectral image classification (HSIC) has achieved impressive progress with emerging studies on deep learning models. However, the classification performance downgrades due to the limited number of annotated samples, especially for minority classes. Notably, the imbalanced data dilemma is familiar in remote sensing hyperspectral image because the ground objects are commonly distributed without evenness. Therefore, this paper proposes a novel deep generative spectral-spatial classifier (DGSSC) for addressing the issues of imbalanced HSIC. Specifically, the DGSSC comprises three components, a two-stage encoder, a decoder, and a classifier, which are trained in an end-to-end manner. In particular, to exploit the abundant spectral-spatial features with relatively low computational complexity, the first stage of the encoder comprises successive three-dimensional (3D) and two-dimensional (2D) convolutions, exploring the spectral-spatial and deep spatial information. In addition, the second stage involves the deep latent variable model to achieve minority-class data augmentation. Furthermore, a patch distance-based reconstruction loss function is designed to facilitate the outputs of the decoder being more similar to the input 3D patch samples. The proposed DGSSC can outperform the state-of-the-art methods on three benchmark datasets, especially with its more robust prediction results. For instance, the DGSSC achieves a remarkable 97.85% mean overall accuracy with 0.24% standard deviation over ten independent runs with randomly selected imbalanced 1% training samples on the University of Pavia dataset.