Moderated resource elasticity for stream processing applications

Publikationstyp
Conference Paper
Date Issued
2018
Sprache
English
Author(s)
Borkowski, Michael  
Hochreiner, Christoph  
Schulte, Stefan  
TORE-URI
http://hdl.handle.net/11420/11888
First published in
Lecture notes in computer science  
Number in series
10659 LNCS
Start Page
5
End Page
16
Citation
European Conference on Parallel Processing (Euro-Par 2017)
Contribution to Conference
European Conference on Parallel Processing, Euro-Par 2017  
Publisher DOI
10.1007/978-3-319-75178-8_1
Scopus ID
2-s2.0-85042478628
In stream processing, elasticity is often realized by adapting the system scale and topology according to the volume of input data. However, this volume is often fluctuating, with a high degree of noise, which can trigger a high amount of scaling operations. Since these scaling operations introduce additional overhead and cost, systems employing such approaches are at risk of spending a significant amount of time scaling up and down, nullifying the positive effects of scalability. To overcome this, we propose an approach for moderating the scaling behavior of stream processing applications by reducing the number of scaling operations, while still providing quick responses to changes in input data volume. Contrary to existing approaches, instead of using linear smoothing techniques, we show how to employ non-linear filtering techniques from the field of signal processing to pre-process the raw volume measurements, mitigating superfluous scaling operations, and effectively reducing the number of such operations by up to 94%.
Subjects
EKF
Elasticity
Stream processing
TVD