Multi-Channel Speech Separation with Cross-Attention and Beamforming

Originally, single-channel source separation gained more
research interest. It resulted in immense progress. Multichannel
(MC) separation comes with new challenges posed by
adverse indoor conditions making it an important field of study.
We seek to combine promising ideas from the two worlds.
First, we build MC models by extending current single-channel
time-domain separators relying on their strength. Our approach
allows reusing pre-trained models by inserting designed
lightweight reference channel attention (RCA) combiner, the
only trained module. It comprises two blocks: the former allows
attending to different parts of other channels w.r.t. the reference
one, and the latter provides an attention-based combination of
channels. Second, like many successful MC models, our system
incorporates beamforming and allows for the fusion of the network
and beamformer outputs. We compare our approach with
the SOTA models on the SMS-WSJ dataset and show better or
similar performance.