Options
Pathfinder: Fault tolerance for stream processing systems
Publikationstyp
Conference Paper
Date Issued
2019-04
Sprache
English
Start Page
29
End Page
39
Article Number
8848250
Citation
5th IEEE International Conference on Big Data Service and Applications (BigDataService 2019)
Contribution to Conference
Publisher DOI
Scopus ID
Stream processing systems are able to integrate data from various sources, and to invoke self-hosted and external operators. In case of faults, such systems usually rely on the redundancy of single stream processing operators, while the relationship between the single operators is not taken into account. Hence, there is a lack of approaches towards fault-tolerant distributed stream processing which consider that stream processing applications are often composed of different operators. Within this paper, we present the Pathfinder framework which overcomes this shortcoming by enabling functional redundancy at the level of stream processing operator paths. During system runtime, Pathfinder reacts to operator failures in the main path by switching to a fault-free path with a similar functionality. To restore the main path once a failed operator has recovered, Pathfinder uses the circuit breaker pattern.
Subjects
Big data
Data streams
Fault tolerance
Stream processing
DDC Class
600: Technik