Thesis Details

Out-of-Vocabulary Words Detection and Recovery

Ph.D. Thesis Student: Egorova Ekaterina Academic Year: 2022/2023 Supervisor: Černocký Jan, prof. Dr. Ing.
Czech title
Detekce a obnova slov mimo slovník
Language
English
Abstract

The thesis explores the field of out-of-vocabulary word (OOV) processing within the task of automatic speech recognition (ASR). It defines the two separate OOV processing tasks - that of detection and recovery - and proposes success metrics for both the tasks. Different approaches to OOV detection and recovery are presented within the frameworks of hybrid and end-to-end (E2E) ASR. These approaches and compared on an open access LibriSpeech database to facilitate replicability.

Hybrid approach uses modified decoding graph with phoneme substrings and utilizes full lattice representations for detection and recovery of recurrent OOVs. Recovered OOVs are added to the dictionary and the language model (LM) to improve ASR system performance. 

The second approach employs inner representations of a word-predicting Listen Attend and Spell architecture (LAS) E2E system to perform OOV detection task. Detection recall and precision rates improved drastically in comparison with the hybrid approach. Recur-rent OOV recovery is performed on a separate character-predicting system with the use of detected time frames and probabilistic clustering.Finally, we propose a new speller architecture with a capability of learning OOV representations together with the word predicting network (WPN) training. The speller forces word embeddings to be spelling-aware during the training and thus not only provides OOV recovery, but also improves the WPN performance.

Keywords

out-of-vocabulary words, automatic speech recognition, hybrid ASR, end-to-end ASR, neural architectures, Listen Attend and Spell.

Department
Degree Programme
Computer Science and Engineering, Field of Study Computer Science and Engineering
Files
Status
defended
Date
16 December 2022
Citation
EGOROVA, Ekaterina. Out-of-Vocabulary Words Detection and Recovery. Brno, 2022. Ph.D. Thesis. Brno University of Technology, Faculty of Information Technology. 2022-12-16. Supervised by Černocký Jan. Available from: https://www.fit.vut.cz/study/phd-thesis/768/
BibTeX
@phdthesis{FITPT768,
    author = "Ekaterina Egorova",
    type = "Ph.D. thesis",
    title = "Out-of-Vocabulary Words Detection and Recovery",
    school = "Brno University of Technology, Faculty of Information Technology",
    year = 2022,
    location = "Brno, CZ",
    language = "english",
    url = "https://www.fit.vut.cz/study/phd-thesis/768/"
}
Back to top