Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization

Combining end-to-end neural speaker diarization (EEND) with
vector clustering (VC), known as EEND-VC, has gained interest
for leveraging the strengths of both methods. EEND-VC
estimates activities and speaker embeddings for all speakers
within an audio chunk and uses VC to associate these activities
with speaker identities across different chunks. EEND-VC
generates thus multiple streams of embeddings, one for each
speaker in a chunk. We can cluster these embeddings using constrained
agglomerative hierarchical clustering (cAHC), ensuring
embeddings from the same chunk belong to different clusters.
This paper introduces an alternative clustering approach, a
multi-stream extension of the successful Bayesian HMM clustering
of x-vectors (VBx), called MS-VBx. Experiments on
three datasets demonstrate that MS-VBx outperforms cAHC in
diarization and speaker counting performance.