Publication Details
Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization
Tawara Naohiro (NTT)
Diez Sánchez Mireia, M.Sc., Ph.D. (DCGM FIT BUT)
Landini Federico Nicolás (DCGM FIT BUT)
Silnova Anna, MSc., Ph.D. (DCGM FIT BUT)
Ogawa Atsunori (NTT)
Nakatani Tomohiro (NTT)
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT)
Araki Shoko (NTT)
speaker diarization, end-to-end, VBx, clustering
Combining end-to-end neural speaker diarization (EEND) with vector clustering (VC), known as EEND-VC, has gained interest for leveraging the strengths of both methods. EEND-VC estimates activities and speaker embeddings for all speakers within an audio chunk and uses VC to associate these activities with speaker identities across different chunks. EEND-VC generates thus multiple streams of embeddings, one for each speaker in a chunk. We can cluster these embeddings using constrained agglomerative hierarchical clustering (cAHC), ensuring embeddings from the same chunk belong to different clusters. This paper introduces an alternative clustering approach, a multi-stream extension of the successful Bayesian HMM clustering of x-vectors (VBx), called MS-VBx. Experiments on three datasets demonstrate that MS-VBx outperforms cAHC in diarization and speaker counting performance.
@INPROCEEDINGS{FITPUB13110,
author = "Marc Delcroix and Naohiro Tawara and Mireia S\'{a}nchez Diez and Nicol\'{a}s Federico Landini and Anna Silnova and Atsunori Ogawa and Tomohiro Nakatani and Luk\'{a}\v{s} Burget and Shoko Araki",
title = "Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization",
pages = "3477--3481",
booktitle = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",
journal = "Proceedings of Interspeech - on-line",
volume = 2023,
number = 08,
year = 2023,
location = "Dublin, IE",
publisher = "International Speech Communication Association",
ISSN = "1990-9772",
doi = "10.21437/Interspeech.2023-628",
language = "english",
url = "https://www.fit.vut.cz/research/publication/13110"
}