Thesis Details
Application of Mean Normalized Stochastic Gradient Descent for Speech Recognition
The artificial neural networks are on the rise in recent years. One possible optimization technique is mean-normalized stochastic gradient descent recently proposes by Wiesler et al. [1]. This work further explains and examines this method on phoneme classification task. Not all findings of Wiesler et al. can be confirmed. The mean-normalized SGD is helpful only if the network is large enough (but not too deep) and if the sigmoid non-linear function is used. Otherwise, the mean-normalized SGD slightly impairs the network performance and therefore cannot be recommended as a general optimization technique.
[1] Simon Wiesler, Alexander Richard, Ralf Schluter, and Hermann Ney.Mean-normalized stochastic gradient for large-scale deep learning. In Acoustics,Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on,pages 180{184. IEEE, 2014.
Neural networks, machine learning, speech recognition, deep learning, stochastic gradient descent.
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT), člen
Kočí Radek, Ing., Ph.D. (DITS FIT BUT), člen
Kotásek Zdeněk, doc. Ing., CSc. (DCSY FIT BUT), člen
Křivka Zbyněk, Ing., Ph.D. (DIFS FIT BUT), člen
@bachelorsthesis{FITBT17867,
author = "Jan Klus\'{a}\v{c}ek",
type = "Bachelor's thesis",
title = "Application of Mean Normalized Stochastic Gradient Descent for Speech Recognition",
school = "Brno University of Technology, Faculty of Information Technology",
year = 2015,
location = "Brno, CZ",
language = "english",
url = "https://www.fit.vut.cz/study/thesis/17867/"
}