Publication Details

Heterogeneity-Aware Scheduler for Stream Processing Frameworks

RYCHLÝ Marek, ŠKODA Petr and SMRŽ Pavel. Heterogeneity-Aware Scheduler for Stream Processing Frameworks. International Journal of Big Data Intelligence, vol. 2, no. 2, pp. 70-80. ISSN 2053-1397. Available from: http://www.inderscience.com/info/inarticle.php?artid=69090
Czech title
Plánovač využívající znalosti heterogenního prostředí pro rámce zpracovávající proudy dat
Type
journal article
Language
english
Authors
URL
Keywords

scheduling; resource awareness; benchmarking; stream processing; Apache Storm; heterogeneous clusters; heterogeneity awareness; resource allocation

Abstract

This article discusses problems and decisions related to scheduling of stream processing applications in heterogeneous clusters. An overview of the current state of the art of the stream processing on heterogeneous clusters with a focus on resource allocation and scheduling is presented first. Then, common scheduling approaches of various stream processing frameworks are discussed and their limited applicability in the heterogeneous environment is demonstrated on a simple stream application. Finally, the article presents a novel heterogeneity-aware scheduler for the stream processing frameworks based on design-time knowledge as well as benchmarking techniques. It is shown that the scheduler overcomes alternatives in resource-aware deployment over cluster nodes and thus it leads to a better utilisation of the clusters.

Published
2015
Pages
70-80
Journal
International Journal of Big Data Intelligence, vol. 2, no. 2, ISSN 2053-1397
Publisher
Inderscience Publishers
DOI
BibTeX
@ARTICLE{FITPUB10729,
   author = "Marek Rychl\'{y} and Petr \v{S}koda and Pavel Smr\v{z}",
   title = "Heterogeneity-Aware Scheduler for Stream Processing Frameworks",
   pages = "70--80",
   journal = "International Journal of Big Data Intelligence",
   volume = 2,
   number = 2,
   year = 2015,
   ISSN = "2053-1397",
   doi = "10.1504/IJBDI.2015.069090",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/10729"
}
Files
Back to top