Thesis Details

Gaussian Processes Based Hyper-Optimization of Neural Networks

Master's Thesis Student: Coufal Martin Academic Year: 2019/2020 Supervisor: Beneš Karel, Ing.
Czech title
Hyper-optimalizace neuronových sítí založená na Gaussovských procesech
Language
English
Abstract

The goal of this thesis is to create a lightweight toolkit for artificial neural network hyper-parameter optimisation. The optimisation toolkit has to be able to optimise multiple, possibly correlated hyper-parameters. I solved this problem by creating an optimiser that uses Gaussian processes to predict the influence of the hyper-parameters on the resulting neural network accuracy. Based on the experiments on multiple benchmark functions, the toolkit is able to provide better results than random search optimisation and thus reduce the number of necessary optimisation steps. The random search optimisation provided better results only in the first few optimisation steps before Gaussian process optimisation creates sufficient model of the problem. However the experiments on MNIST dataset show that random optimisation achieves almost always better results than used GP optimiser. These differences between the experiments results are probably caused by insufficient complexity of the benchmarks or by selected parameters of the implemented optimiser.

Keywords

hyper-parameter tuning, Gaussian processes, neural networks optimisation, regression problem solving, kernels

Department
Degree Programme
Information Technology, Field of Study Intelligent Systems
Files
Status
defended, grade A
Date
17 July 2020
Reviewer
Committee
Herout Adam, prof. Ing., Ph.D. (DCGM FIT BUT), předseda
Beran Vítězslav, doc. Ing., Ph.D. (DCGM FIT BUT), člen
Burget Lukáš, doc. Ing., Ph.D. (DCGM FIT BUT), člen
Hradiš Michal, Ing., Ph.D. (DCGM FIT BUT), člen
Křivka Zbyněk, Ing., Ph.D. (DIFS FIT BUT), člen
Orság Filip, Ing., Ph.D. (DITS FIT BUT), člen
Citation
COUFAL, Martin. Gaussian Processes Based Hyper-Optimization of Neural Networks. Brno, 2020. Master's Thesis. Brno University of Technology, Faculty of Information Technology. 2020-07-17. Supervised by Beneš Karel. Available from: https://www.fit.vut.cz/study/thesis/22368/
BibTeX
@mastersthesis{FITMT22368,
    author = "Martin Coufal",
    type = "Master's thesis",
    title = "Gaussian Processes Based Hyper-Optimization of Neural Networks",
    school = "Brno University of Technology, Faculty of Information Technology",
    year = 2020,
    location = "Brno, CZ",
    language = "english",
    url = "https://www.fit.vut.cz/study/thesis/22368/"
}
Back to top