Detail výsledku

Spatially Aware Self-Supervised Models for Multi-Channel Neural Speaker Diarization

HAN, J.; WANG, R.; MASUYAMA, Y.; DELCROIX, M.; ROHDIN, J.; DU, J.; BURGET, L. Spatially Aware Self-Supervised Models for Multi-Channel Neural Speaker Diarization. ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona, Španělské království: IEEE, 2026. p. 17447.ISBN: 979-8-3315-6701-9.
Typ
článek ve sborníku konference
Jazyk
angličtina
Autoři
Han Jiangyu, UPGM (FIT)
Wang Ruoyu
Masuyama Yoshiki
Delcroix Marc
Rohdin Johan Andréas, M.Sc., Ph.D., FIT (FIT), UPGM (FIT)
Du Jun
Burget Lukáš, doc. Ing., Ph.D., UPGM (FIT)
Abstrakt

Self-supervised models such as WavLM have demonstrated strong performance for neural speaker diarization. However, these models are typically pre-trained on single-channel recordings, limiting their effectiveness in multi-channel scenarios. Existing diarization systems built on these models often rely on DOVER-Lap to combine outputs from individual channels. Although effective, this approach incurs substantial computational overhead and fails to fully exploit spatial information. In this work, building on DiariZen, a pipeline that combines WavLM-based local end-to-end neural diarization with speaker embedding clustering, we introduce a lightweight approach to make pre-trained WavLM spatially aware by inserting channel communication modules into the early layers. Our method is agnostic to both the number of microphone channels and array topologies, ensuring broad applicability. We further propose to fuse multi-channel speaker embeddings by leveraging spatial attention weights. Evaluations on five public datasets show consistent improvements over single-channel baselines and demonstrate superior performance and efficiency compared with DOVER-Lap. Our source code is publicly available at https://github.com/BUTSpeechFIT/DiariZen.

Klíčová slova

Multi-channel speaker diarization, DiariZen, self-supervised, WavLM, cross-channel communication

URL
Rok
2026
Strany
17447–17451
Sborník
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Konference
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
ISBN
979-8-3315-6701-9
Vydavatel
IEEE
Místo
Barcelona, Španělské království
DOI
BibTeX
@inproceedings{BUT212033,
  author="Jiangyu {Han} and  {} and  {} and  {} and Johan Andréas {Rohdin} and  {} and Lukáš {Burget}",
  title="Spatially Aware Self-Supervised Models for Multi-Channel Neural Speaker Diarization",
  booktitle="ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",
  year="2026",
  pages="17447--17451",
  publisher="IEEE",
  address="Barcelona, Španělské království",
  doi="10.1109/icassp55912.2026.11463023",
  isbn="979-8-3315-6701-9",
  url="https://ieeexplore.ieee.org/document/11463023"
}
Soubory
Projekty
Jazykověda, umělá inteligence a jazykové a řečové technologie: od výzkumu k aplikacím, EU, MEZISEKTOROVÁ SPOLUPRÁCE, EH23_020/0008518, zahájení: 2025-01-01, ukončení: 2028-12-31, řešení
Výzkumné skupiny
Pracoviště
Nahoru