Discriminative Training of VBx Diarization

Bayesian HMM clustering of x-vector sequences (VBx) has be-
come a widely adopted diarization baseline model in publications
and challenges. It uses an HMM to model speaker turns, a gen-
eratively trained probabilistic linear discriminant analysis (PLDA)
for speaker distribution modeling, and Bayesian inference to esti-
mate the assignment of x-vectors to speakers. This paper presents
a new framework for updating the VBx parameters using discrim-
inative training, which directly optimizes a predefined loss. We
also propose a new loss that better correlates with the diarization
error rate compared to binary cross-entropy - the default choice
for diarization end-to-end systems. Proof-of-concept results across
three datasets (AMI, CALLHOME, and DIHARD II) demonstrate
the method's capability of automatically finding hyperparameters,
achieving comparable performance to those found by extensive grid
search, which typically requires additional hyperparameter behavior
knowledge. Moreover, we show that discriminative fine-tuning of
PLDA can further improve the model's performance. We release the
source code with this publication.