Result Details

Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text

BASKAR, M.; WATANABE, S.; ASTUDILLO, R.; HORI, T.; BURGET, L.; ČERNOCKÝ, J. Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text. In Proceedings of Interspeech. Proceedings of Interspeech. Graz: International Speech Communication Association, 2019. no. 9, p. 3790-3794. ISSN: 1990-9772.
Type
conference paper
Language
English
Authors
Baskar Murali Karthick, Ing., Ph.D., DCGM (FIT)
Watanabe Shinji, FIT (FIT)
ASTUDILLO, R.
HORI, T.
Burget Lukáš, doc. Ing., Ph.D., DCGM (FIT)
Černocký Jan, prof. Dr. Ing., DCGM (FIT)
Abstract

Sequence-to-sequence automatic speech recognition (ASR)models require large quantities of data to attain highperformance. For this reason, there has been a recent surgein interest for unsupervised and semi-supervised training insuch models. This work builds upon recent results showingnotable improvements in semi-supervised training usingcycle-consistency and related techniques. Such techniquesderive training procedures and losses able to leverage unpairedspeech and/or text data by combining ASR with Text-to-Speech(TTS) models. In particular, this work proposes a newsemi-supervised loss combining an end-to-end differentiableASR!TTS loss with TTS!ASR loss. The method is ableto leverage both unpaired speech and text data to outperformrecently proposed related techniques in terms of %WER. Weprovide extensive results analyzing the impact of data quantityand speech and text modalities and show consistent gains acrossWSJ and Librispeech corpora. Our code is provided in ESPnetto reproduce the experiments.

Keywords

Sequence-to-sequence, end-to-end, ASR, TTS,semi-supervised, unsupervised, cycle consistency

URL
Published
2019
Pages
3790–3794
Journal
Proceedings of Interspeech, vol. 2019, no. 9, ISSN 1990-9772
Proceedings
Proceedings of Interspeech
Conference
Interspeech Conference
Publisher
International Speech Communication Association
Place
Graz
DOI
10.21437/Interspeech.2019-3167
UT WoS
000831796403198
EID Scopus
2-s2.0-85074698215
BibTeX 
@inproceedings{BUT159996,
  author="BASKAR, M. and WATANABE, S. and ASTUDILLO, R. and HORI, T. and BURGET, L. and ČERNOCKÝ, J.",
  title="Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text",
  booktitle="Proceedings of Interspeech",
  year="2019",
  journal="Proceedings of Interspeech",
  volume="2019",
  number="9",
  pages="3790--3794",
  publisher="International Speech Communication Association",
  address="Graz",
  doi="10.21437/Interspeech.2019-3167",
  issn="1990-9772",
  url="https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3167.pdf"
}
Files
Projects
DARPA Low Resource Languages for Emergent Incidents (LORELEI) - Exploiting Language Information for Situational Awareness (ELISA), University of Southern California, start: 2015-09-01, end: 2020-03-31, completed
IARPA Machine Translation for English Retrieval of Information in Any Language (MATERIAL) - Foreign Language Automated Information Retrieval (FLAIR), IARPA, start: 2017-09-21, end: 2021-10-22, completed
IT4Innovations excellence in science, MŠMT, Národní program udržitelnosti II, LQ1602, start: 2016-01-01, end: 2020-12-31, completed
Zpracování, zobrazování a analýza multimediálních a 3D dat, BUT, Vnitřní projekty VUT, FIT-S-17-3984, start: 2017-03-01, end: 2020-02-29, completed
Research groups
Speech Data Mining Research Group BUT Speech@FIT (RG SPEECH)
Departments
Department of Computer Graphics and Multimedia (DCGM)
Back to top