Unsupervised Mineral Segmentation with Graph Neural Networks and Multi-modal SEM Data

We propose a novel method for multi-modal mineral segmentation that utilises backscattered electron (BSE) images and sparse Energy-Dispersive X-ray spectroscopy (EDS) measurements from Scanning Electron Microscope (SEM). The method uses Graph Neural Networks for simultaneous data fusion and segmentation. The segmentation is unsupervised, allowing for the separation of mineral phases even if they were not included in the training dataset. The segments are created from graph structure, where each BSE pixel is connected to a set of EDS nodes that correspond to pointwise spectral measurements. This connection (edge in the graph) is perceived as a choice, allowing the network to select an EDS measurement to which the BSE pixel most likely belongs. Each pixel is assigned to an EDS measurement, effectively creating segments; inside of each is exactly one EDS measurement. This allows for unsupervised segmentation applicable to any mineral phase. In our experiments with challenging mineral datasets, we show that the proposed method outperforms state-of-the-art segmentation accuracy while scaling more efficiently with sample size.