Spectral–Spatial MLP Like Network With Reciprocal Points Learning for Open Set Hyperspectral Image C

In recent years, deep-learning-based hyperspectral image (HSI) classification methods have achieved significant development and gradually become widely applied. The existing advanced methods can achieve near-saturation performance with sufficient labels in a closed-set environment (CSE); i.e., training set and test set are all known categories of ground objects. However, the real world is usually open because of the diversity of land covers; i.e., test set exists unknown categories that are not labeled in the training set. Therefore, the prevalent advanced CSE methods still cannot effectively and robustly handle unknown categories of ground objects in an open-set environment (OSE). Therefore, we propose a spectral–spatial multiple layer perceptron (MLP)-like network with reciprocal points learning (SSMLP-RPL) to improve the performance of open-set HSI classification. First, a feature learning framework based on RPL is constructed to model the extra-category space and reduce the risk of open space. The learned feature space enables to enlarge the distance between the known and unknown categories. Besides, we further propose to utilize a learnable dynamic threshold of each known category to effectively distinguish the unknown categories and improve open performance of the model. Second, to enhance the capacity of feature learning, an SSMLP-like network is designed to capture the spectral–spatial feature merely with a series of fully connected (FC) layers, which mainly involve SpeFC and SpaFC two modules. Among them, the SpaFC module enables to model spacial semantics, and the SpeFC module enables to model long-distance spectral dependence. Extensive experiments on three benchmark HSIs show that SSMLP-RPL has a competitive performance both in CSE and OSE and even surpasses currently advanced closed-set and open-set HSI classification methods. As an end-to-end HSI classification framework of MLP backbone, SSMLP network can compete with the advanced works based on CNN and Transformer.