Recognition of Multi-Talker Overlapping Speech Using Neural Networks

This work deals with the speech recognition of overlapping speakers using a neural network. It examines the problem of speech recognition from multiple speakers and the ways in which this problem is solved. Specifically, in addition to traditional components such as convolutional neural networks, LSTM, etc., it is also an application of special components: attention mechanism and gated convolution. And also the application of a technique called permutation invariant training. Part of this work is to apply these approaches to assigned training data, which consists of artificially created mixtures of two speakers reading articles from the Wall Street Journal. The next step was to train the respective architectures using the combinations of the elements mentioned above. The models in this work replace the acoustic model. There were two architectures using different types of attention mechanism and one without it. Experiments have shown that architectures using the attention mechanism in this type of task have not surpassed more traditional architecture by suffering from gated convolution. Nevertheless, they showed potential.

speech recognition,neural networks,attention mechanism,overlapping speech