Result Details
Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks
Reich Bořek, Ing., DCGM (FIT)
Zolotarev Fedor
Eerola Tuomas, Prof., FIT (FIT)
Zemčík Pavel, prof. Dr. Ing., dr. h. c., DCGM (FIT)
We propose a novel Graph Neural Network-based method for segmentation based on
data fusion of multimodal Scanning Electron Microscope (SEM) images. In most
cases, Backscattered Electron (BSE) images obtained using SEM do not contain
sufficient information for mineral segmentation. Therefore, imaging is often
complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral
measurements that provide highly accurate information about the chemical
composition but that are time-consuming to acquire. This motivates the use of
sparse spectral data in conjunction with BSE images for mineral segmentation. The
unstructured nature of the spectral data makes most traditional image fusion
techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks
to fuse the two modalities and segment the mineral phases simultaneously. Our
results demonstrate that providing EDS data for as few as 1% of BSE pixels
produces accurate segmentation, enabling rapid analysis of mineral samples. The
proposed data fusion pipeline is versatile and can be adapted to other domains
that involve image data and point-wise measurements.
Graph neural networks; Data fusion; Mineral segmentation; Scanning electron
microscope
@article{BUT197861,
author="Samuel {Repka} and Bořek {Reich} and {} and Tuomas {Eerola} and Pavel {Zemčík}",
title="Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks",
journal="Pattern Recognition Letters",
year="2025",
volume="193",
number="193",
pages="79--85",
doi="10.1016/j.patrec.2025.04.012",
issn="0167-8655",
url="https://doi.org/10.1016/j.patrec.2025.04.012"
}